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PREFACE 


The document contains the proceedings of the training workshop on Innovative 
Design of Complex Engineering Systems. The workshop was held at the Peninsula 
Higher Education Center, Hampton, Virginia, March 23 and 24, 2004. The workshop 
was jointly sponsored by Old Dominion University and NASA. Workshop attendees 
came from NASA, other government agencies, industry and universities. The objectives 
of the workshop were to a) provide broad overviews of the diverse activities related to 
innovative design of high-tech engineering systems; and b) identify training needs for 
future aerospace work force development in the design area. The format of the workshop 
included fifteen, half-hour overview-type presentations, a panel discussion on how to 
teach and train engineers in innovative design, and three exhibits by co mm ercial vendors. 


A hm ed K. Noor 
Old Dominion University 
Center for Advanced Engineering Environments 

Hampton, Virginia 
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Outline 


Economic stresses and a very competitive market are forcing many industries to 
reduce cost and development time, and to insert emerging technologies into their 
products. Engineers are asked to design faster, ever more complex systems. They must 
find globally optimal designs that take uncertainties and risk into consideration. 

Over the last few years, a number of methodologies and technologies have been 
developed and utilized to support these efforts. Also, a number of approaches have been 
proposed for design education and training. 

An attempt is made in this presentation to give a broad overview of the activities 
on innovative design and to set the stage for succeeding presentations. The presentation 
is divided into four parts (Figure 1). In the first part, examples of future aerospace 
systems are given, along with some of their major characteristics and enabling 
technologies. The second part provides a brief overview of some of the current activities 
on innovative design of complex engineering systems. The third part describes a vision 
for future innovative design along with the key components of the innovative design 
process. The fourth part lists the objectives of the workshop and some of the sources of 
information on innovative design. 
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Examples of Future Aerospace Systems and Some of their Characteristics 


The realization of NASA’s ambitious space exploration initiative with the current 
budget constraints will require new kinds of aerospace systems and missions that use 
novel technologies and manage risks in new ways. Future aerospace systems must be 
autonomous, evoluable, resilient and highly distributed. Two examples are given in 
Figure 2. The first is a crew exploration vehicle. The second is a lunar outpost. Each of 
these is a complex system of systems. 
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Enabling Technologies for Future Aerospace Systems 


The characteristics of future aerospace systems identified in Figure 2 are highly 
coupled and their realization requires the synergistic coupling of the revolutionary and 
other leading-edge technologies listed in Figure 3. The four revolutionary technologies 
are nanotechnology, biotechnology, information / knowledge technology, and cognitive 
systems technology. The other leading-edge technologies are high-productivity 
computing; high-capacity communication; modeling, simulation and visualization; virtual 
product development; intelligent software agents; reliability / risk management; human- 
computer symbiosis; and human performance. 



[Capacity 


“Human 

Performance, 


Technology 


Technology 


*Human- 

Computer- 

Symbiosis] 


Information/ 

Knowledge 

Technology 


Figure 3 


5 







Definitions of Engineering Design 


Although there is no single universally acceptable definition of engineering 
design, Figure 4 shows the definition given by the Accreditation Board for Engineering 
and Technology (ABET), along with four other definitions. Design is often equated with 
Synthesis and with the Practice of the engineering profession. In engineering curricula, 
Design is differentiated from Science, Analysis, and Theory. 

Design is concerned with synthesis of information into a whole, with the everyday 
world of engineering practice, and with problems that cross discipline boundaries. 
Science, analysis, and theory achieve their power through simplifications and narrowing, 
through research under controlled condition, and by operating within separate disciplines. 
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Design Paradigms 


A number of different design paradigms are currently being used, including the 
four shown in Figure 5 and described subsequently. These are: 

• Design for Safety 

Intended to mitigate risk by improving the resilience of the system to unforeseen 
events. It aims at making the system smart to adapt to changes and self-heal from 
damage. It provides a final barrier against any system degradation and rare unforeseen 
events. 

• Design for Cost and Quality 

This includes Taguchi’s robust design approach. It aims at determining the 
optimum configuration of design parameters for performance, quality and cost. 

• Design for the Life Cycle 

Based on early consideration of several life cycle factors in the design process, 
including testability / inspectability, reliability / availability, maintainability / 
serviceability, upgradeability, safety, and human factors. 

• Design for manufacturing 

Covers all aspects of design and manufacturing integration, including integrated 
assembly design and planning, part design and process planning integration, and robust 
design and variation management. It encompasses design for mass customization, 
layered manufacturing, and remanufacture. 



Figure 5 
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Enhancing Design Performance 


Traditionally, the evaluation of design performance has focused on the outcome 
of the design process, the product. Recently, the human dimensions of designing, namely 
design cognition and human-centered perspective have been added to the design 
performance (Figure 6). 

The design process involves understanding, synthesizing and applying principles 
associated with basic and engineering science for creating new technologies that enable 
new products which satisfy and delight users. 

Design cognition refers to design thinking, i.e., the thought process employed in 
the design. 
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Activities on Innovative Design 


The need for developing new approaches to design, analysis, testing and 
manufacturing of complex engineering systems have been recognized by government 
agencies, industry, and Academia. Examples of projects initiated by these organizations 
are given in Figure 7. 

Among the government projects are the NASA Engineering Training (NET) 
innovative design project, NIST design repository project, DARPA’s Rapid Design 
Exploration and Optimization (RaDEO) project, and the two NSF Projects — Engineering 
Design, and Transferable Integrated Design Engineering Education (TIDEE) projects. 
Several i nn ovative design projects have been initiated by industry groups, including 
Boeing Phantom Works, Lockheed Skunk Works, General Motors (Virtual Vehicle 
Project) and IBM (virtual product innovation project). 

Universities have developed new approaches, laboratories and centers for design 
research and education. Examples include MIT’s Conceive-Design-Implement-Operate 
(CDIO); Georgia Tech Aerospace Systems Design Lab (ASDL); Stanford Center for 
Design Education. 

Also, consortia of universities and other organizations have been formed. An 
example is the Space Systems, Policy and Architecture Research Consortium (SSPARC). 
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Forces Driving a Change in the Design of Complex Systems 


After decades of evolutionary change, revolutionary changes are both needed and 
possible in the design of complex engineering systems. The changes are driven by four 
categories of forces (Figure 8): 

— Changes in High-Tech Organizations. 

Quality was the focus of high-tech organizations in the 1980s. In the 1990s, the move 
from the industrial to the knowledge era shifted the focus to re-engineering and 
streamlining the processes, and then to managing knowledge and creation of high- 
performance workplaces. In the future there is likely to be a move to the biological and 
advanced materials era (referred to as the bioterials era). The focus of high-tech 
organizations will shift to explorations in the cellular and subatomic universe - 
architecting matter. Facilities will be developed for temporal compression and global 
diffusion. 


— Economic and Business Pressures. 

Economic stresses and customer demands for cheaper, better, faster products have driven 
high tech organizations from mass production to mass customization, and to the adoption 
of lean production system concepts. They have integrated simulation and design tools 
with other tools and facilities for lean engineering, manufacturing and supplier 
management. 

— Paradigm Change in Human / Machine / Network Interaction. 

Ubiquitous / pervasive computing and wireless connectivity among diverse teams and 
embedded devices, including thousands of embedded nanodevices per person, will 
become the norm. Consequently, there is a move from human-centered (interactive) 
computing to human-supervised (proactive) computing. Multimodal perceptual, neural 
and other advanced interfaces, which integrate adaptive interfaces with intelligent agents, 
will become available. 

-- Impact of Advances in Technology. 

The synergistic coupling of several leading edge technologies will have a 
significant impact on future products and engineering systems. To realize the potential of 
this syngerism, high-tech organizations will have to provide effective diverse team 
collaboration facilities and interdisciplinary research and development networks (VPD 
hubs). Modeling, simulation and visualization tools will be thought of as network 
services. 
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Forces Driving a Change in the Design of Complex Systems (cont’d) 
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Innovation and Creativity 


Creativity refers to coming up with new ideas. Innovation equals creativity plus 
successful implementation, or putting ideas into practice, which includes idea selection, 
development, and co mm ercialization (Figure 9). Achieving implementation involves 
development of processes, procedures and structures that allow timely and effective 
execution of projects. 

Alliance of technology and creative practices can lead to innovative product 
design. This includes providing new tools and media for designers, and providing 
opportunities to develop creative critical thinking skills. 
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Key Components of Innovative Design Process 

The essential components of the innovative design process can be grouped into 
three categories: virtual product hub, intelligent integrated networked design 
environments, and tools for managing complexities and uncertainties (Figure 10). The 
three categories are described subsequently. 



Figure 10 
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Virtual Product Hub 


Product innovation requires a unique blend of people, processes and technologies. 
All rely on a common capability to collaborate, integrate and innovate: the pervasive use 
of a virtual product hub (Figure 11). The hub incorporates a product life cycle 
management (PLM) system. Modeling, simulation, visualization and optimization tools 
will be thought of as network services. 



Figure 11 
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Intelligent Integrated Networked Design Environment 

Future design environment will provide multidisciplinary and interdisciplinary 
teams with flexible dynamic information devices, novel multiuser displays, intelligent 
software agents, telepresence and other distributed collaboration facilities and multimodal 
interfaces. 

It will exploit information / knowledge and other leading edge technologies to 
facilitate simultaneous collaborative design (across disciplines, tools and organizations); 
automate non-creative tasks; and enable informed design decisions early in the design 
cycle using elaborate knowledge repository, lessons learned and inverse engineering 
(Figure 12). 



Figure 12 
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Tools for Managing Complexities and Uncertainties 


Variety of tools can help designers in managing the complexities and 
uncertainties of future high-tech engineering systems, involving large number of 
interactions among components. These include (Figure 13): 

— Tools for handling complex multiphysics data and varying degrees of model 

fidelity 


— Tools for computational steering (interactively controlling the computational 
process during its execution), inverse steering (where the user specifies the desired 
simulation result, and the system searches for the simulation parameters that achieve this 
result). 


— Emergent synthesis tools for handling hierarchical complexity. These are 
interdisciplinary tools with strong connection to the fields of artificial life, artificial 
intelligence, evolutionary and emergent computation, soft computing, complex adaptive 
systems, reinforcement learning, self organization and others. 



and Uncertainties 



Figure 13 
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Innovative Design Network 


The realization of the full potential of design innovations in the development of 
future complex systems requires, among other things, the establishment of innovative 
design networks. The networks connect diverse, geographically dispersed teams from 
NASA, other government labs, university consortia, industry, technology providers, and 
professional societies (Figure 14). 



Figure 14 
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Product Life-Cycle Simulation 


There is a pressing need to be able to optimize complex engineering systems for 
all aspects of life-cycle performance — including factors such as maintenance, reliability, 
training, and end-of-life disposition (e.g., recycle and disposal). By providing the 
capability to accurately model and simulate all aspects of the product life cycle from the 
earliest stages of mission requirements and concept selection to manufacturing, assembly 
planning and prototyping, testing, operations, maintenance and repairs, organizations can 
significantly reduce costs of acquisition and ownership, and dramatically improve 
operational performances and efficiency (Figure 15). Current development in this area is 
focused largely on CAD-based product life cycle management (PLM) tools. 

Current efforts aim at having integrated models for driving, enabling and 
supporting all phases of the product life cycle. All activities in the life-cycle simulation 
apply and support a central product “meta-model” that is linked to analytical simulation 
tools for design, systems engineering, and decision support; and to all processes, systems, 
and participants in the product life cycle. 
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Product Life-Cycle Simulation (cont’d) 
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Virtual Product Development Hub 

Figure 16 shows the major components of a virtual product hub. These are: 

— Blended virtual product development environment consisting of modeling, life- 
cycle simulation, visualization, and optimization tools (network facilities) 

— Product life cycle management system, incorporating model management, 
product data management, and simulation management 

— Knowledge repository incorporating information about previous projects 
performed by the enterprise 

Collaboration infrastructure for synchronous and asynchronous 
communication, information sharing and group distributed developments 

— Multimodal and advanced interfaces 

— VPD adviser (intelligent software agents) 

The latter three are described subsequently 
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Virtual Product Development Hub 
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Collaboration Infrastructure 


Intelligent software agents (human-like avatars) are used to carry out all the 
routine tasks that can be automated for distributed group collaboration (Figure 17). 
These include scheduling and starting a group meetings; query and display of 
information; and recording the session for the team members who cannot join the 
meeting. 



Figure 17 
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Multimodal and Advanced Interfaces 


Although the WIMP (windows, icons, menus, pointing devices) paradigm has 
provided a stable and global interface, it will not scale to match the myriad form factors 
and uses of platforms in the future collaborative distributed environments. The 
combination of neural, affective, perceptual interfaces and handheld devices will enable 
the interaction with the virtual product hub in more human- like ways (Figure 18). 



Figure 18 
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Virtual Product Development Advisor 


Intelligent software agents (human-like avatars) are used in the VPD hub as 
virtual technical assistants. They provide assistance in the use of the different tools and 
facilities of the hub. This is accomplished by coupling natural language processing, and 
rule-based expert systems with the avatars (Figure 19). 



Figure 19 
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Intelligent Design Environment 


The future design environment will enable collaborative distributed synthesis to 
be performed by geographically dispersed interdisciplinary / multidisciplinary teams. It 
will include flexible and dynamic roomware (active spaces / collaboration landscape) 
facilities consisting of (Figure 20): 

— Portable and stationary information devices 

— Novel multiuser smart displays 

— Telepresence and other distributed collaboration facilities 
— Novel forms of multimodal human / network interfaces 
— Middleware infrastructures and intelligent software agents 
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Intelligent Design Environment (con’t) 
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Objectives and Format of Workshop 


The objectives of the workshop are to (Figure 21): a) provide a broad overview of 
the diverse activities related to innovative design of complex engineering systems; and b) 
identify training needs for the future aerospace workforce development in the design 
area. 


The format included 15 presentations in six sessions. A panel session was 
devoted to “how to teach and train engineers in innovative design.” Three exhibits were 
also organized by technology providers at the meeting. 


OBIECTIUES T ANP FORMAT OF WORKSHOP 
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INFORMATION ON INNOVATIVE DESIGN OF 
COMPLEX ENGINEERING SYSTEMS 

A short list of books, monographs, conference proceedings, survey papers and 

websites on innovative design of complex engineering systems is given subsequently. 

Books, Monographs, and Conference Proceedings : 

[ 1 ] Eris, Ozgur, Effective Inquiry for Innovative Engineering Design, 

Kluwer Academic Publishers, 2004. 

[2] Antonsson, Erik K., and Cagan, Jonathan (editors), Formal Engineering 
Design Synthesis, Cambridge University Press, 2001. 

[3] Kroll, Ehud, Jansson, David G., Condoor, Sridhar, S., Innovative 
Conceptual Design: Theory and Application of Parameter Analysis, 
Cambridge University Press, 2001. 

[4] Tong, Christopher, and Sriram, Duvvuru (editors), Artificial Intelligence 
in Engineering Design: Models of Innovative Design, Reasoning about 
Physical Systems, and Reasoning about Geometry, Academic Press, 

2000. 

[5] Pugh, Stuart, Creating Innovative Products Using Total Design, Prentice 
Hall, 1996. 

Survey Papers and Articles : 

[1] Eris, Ozgur, Leifer, Larry, “Facilitating Product Development 
Knowledge Acquisition: Interaction between the Expert and the Team,” 
International Journal of Engineering Education, Vol 19, No. 1, pp. 142- 
152,2003. 

[2] Klein, Mark, Faratin, Peyman, Sayama, Hiroki, and Bar-Mar, Yaneer, 

“What Complex Systems Research Can Teach Us about Collaborative 
Design,” Proceedings of the Sixth International Conference on Computer 
Supported Cooperative Work Design (CSCWD-2001), IEEE Press, pp. 
5-12,2001. 

[3] Raju, P.K., Sankar, Chetan S., Halpin, Gerald, Halpin, Glennelle, “An 
Innovative Teaching Method to Improve Engineering Design 
Education,” American Society for Engineering Education, St. Louis, 

MO, June 2000. 

[4] Cowan, F. Scott, Allen, Janet K., and Mistree, Farrokh, “Exploring 
Perspectives with Livings Systems Theory in the Design of Complex 
Engineering Systems,” Proceedings of the 44th Annual Conference of 
the International Society for the Systems Sciences, (Allen, J. K., and 
Wilby, J., eds.), ISSS, July 16 - 22, 2000, Toronto, Canada, Paper No. 

20138 

[5] Szykman, Simon, Sriram Ram D., Bochenel, Christophe, Racz, Janusz, 

“The NIST Design Repository,” Soft Computing Engineering Design and 
Manufacturing, July, pp. 5-19, 1998. 
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Websites: 


1. Aerospace System Design Lab, Georgia Tech 
http://www.asdl.gatech.edu 

2. Center for Design Research, Department of Mechanical Engineering, 

Stanford University 

http://www-cdr.stanford.edu 

3. Conceive-Design-Implement-Operate, Department of Aeronautics and 
Astronautics, Massachusetts Institute of Technology 
http://www.cdio.org/index.html 

4. Center for Design Education, Harvey Mudd College 
http://www2.hmc. edu/~dym/CDE_index.html 

5. Chalmers Innovative Design 
http://www.design.chalmers.se/about_us/idsummary.html 

6. The Institute of Systems Research, University of Maryland 
http://www.isr.umd.edu/ISR/about/defmese.html 

7. Rapid Design Exploration and Optimization (RaDEO) 
http:// www.darpa.mil/dso/trans/swo.htm 

8. National Institute Standards and Technology Design Repository (Virtual 
Library) 

http://nvl.nist.gov/ 

9. National Science Foundation Engineering Design 
http://www.nsf.gov/home/eng/ 

10. National Science Foundation, Transferable Integrated Design 
Engineering Education (TIDEE) 
http://www.tidee.cea.wsu.edu 

1 1 . IBM Virtual Product Innovation 

http : //www 1 . ibm. com/ industries/ automotive/ doc/ content/component/services/2 83 
660108.html 
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To develop an advanced technology aerospace product on budget and on schedule, 
data indicates that what I am labeling a “Product Development Imperative” exists. This 
presentation discusses that imperative and shows how critical capabilities have been 
developed in the Robust Design Computational System and are being developed in the 
Accelerated Insertion of Materials program. 
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A chart from a NASA study, published in Aerospace America, illustrates the 
economic issues in deciding to invest in an access to space capability. For each option 
shown, an up-front investment is required to achieve a desired benefit. Generally, the 
greater the desired benefit, the more investment is required. In the private sector, a 
financial analyst would compute a return on investment or an internal rate of return to 
assess the worth of the investment. Government agencies may or may not use such an 
analysis, but to justify investing, at some point a decision is made that the benefit of some 
option is worth the investment. If the size of the required cost increases, if the schedule 
increases, or if the benefit is smaller than planned, the cost-benefit analysis associated 
with the investment may be compromised. For development programs, the issue is how 
to develop an advanced technology product on a planned budget, on a planned schedule, 
and achieve the targeted goals. 


NASA Study Illustrates Economic Basis of 
Access to Space Decisions 



Current Systems (Life-cycle cost-$233B) 
Option 1 (LCC-$230B| 

Option 2 (LCC=$192B) 

Option 3A (LCC-$198B) 

Option 3B (ICC-$!69B) 


Bekey, Ivan: Powell, Richard; Austin, Robert. “NASA studies access to space.” Aerospace America, 
May, 1994. PP 38 - 43 
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Often the consequence of exceeding the budget or schedule or not achieving the 
planned goal on a government funded program is that the program is cancelled. Shown 
in one company’s (not Boeing, although the Boeing experience would be similar) 
experience in government- funded launch vehicle programs over the last 15 years. Note 
that none of the programs over the last 15 years ever reached flight status. What has been 
the return on the government investment in these programs? 


Government Funded Launch Vehicle Programs 
Over the Last 15 Years 

One Company’s Experience 



Preliminary Component Demonstration Protoqual Qualification Full Scale Flight Production 

Design Development Testing Development 
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To understand what is happening, we start with the initial investment phase. The cost 
profile shown is taken from a NASA-funded cost estimation contract which looked at the 
historical experience of advanced technology rocket engines. What stands out clearly is 
that 73 percent of the cost of developing an engine to the point of single engine 
certification was absorbed in corrective actions on full-scale hardware after it had been 
designed and installed in the test stand. The large cost for corrective actions represents a 
cost overrun. A second set of data characterizing both advanced technology rocket 
engines and advanced technology jet engines shows a different breakout. To see how 
these two sets of data help understand what is happening in the product development 
process, we start with a simple depiction of a development process. 


Development Cost Dominated by Rework Cycles 
After Full Scale Engine Testing Begins 
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Corrective Actions Largest Development Cost 


Initial Design 
Engineering 
Demonstration 

Corrective Actions on Test Stand 


2 percent □ 
15 percent □ 


10 percent | | 

73 percent 


Development Costs by Discipline 


Engineering & Mgmt 
Test 

Hardware 


25 percent 
20-25 percent 
50-55 percent 


NASA cost 
analysis contract 
for rocket 
engines 


NASA consortium 
data for jet and 
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A simplified product development process is shown consisting of design, 
manufacture, test (if necessary) and customer use. If problems with the design are found 
in manufacturing, changes may be made to the product to resolve the manufacturing 
problems. This is the famous “throw it over the wall” problem, and one measure 
appropriate for this “Producibility” cycle is the unit cost. If problems with the design are 
found during tests of the full scale product, then the “Test-Fail-Fix” cycle occurs. For 
advanced technology propulsion systems, both jet engine and rocket engine, the typical 
number of rework cycles and the cost of a single rework cycle are shown. Taking the 
midpoints, 200 rework cycles and $10 million per rework cycle, gives a $2 billion result. 
Clearly, the Test-Fail-Fix cycle has been a major cost element in development programs. 
If a problem is found when the product is in customer use, the Operabilty Cycle could 
occur to solve the problem with a design change. Change at this stage is so expensive, 
however, that when it occurs, it occurs in blocks (groups) of changes. Often, the 
consequence is increased maintenance costs or limitations on product use rather that incur 
the costs of the Operability Cycle. 


Simplified Product Development Process 
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To meet the time limit for this presentation, several steps have been skipped. When it 
became clear that the cost of rework cycles dominated development costs for advanced 
technology propulsion systems, further investigations suggested that rework cycles were 
in turn possibly determined by the level of risk assumed when the decision was made to 
design the hardware. Accordingly, some criteria, based on an Air Force risk assessment 
guide, were developed to assess the level of risk assumed on some heritage products. 
Note that the criteria are similar to the NASA Technology Readiness Level scale for the 
case shown, but the numbers are two decimal place numbers between 0 and 1 rather than 
the levels of 1 to 9. 


Criteria Developed to Assess Technical Uncertainty 


COMPLEXITY 

DESIGN MATURITY 


ENVIRONMENT 


TECHNOLOGY 


0.01 Technology operational and deployed. 

0.2 Technology successfully in use on another mature program. 

0.3 Technology successfully tested in operational environment. 

0.5 Technology successfully tested in relevant environment. 

0.7 Proof-of-concept experiments successfully completed. 

0.9 Relevant combinations of basic mechanisms analyzed, tested, and validated 

0.95 Basic research only. No development work 
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Based on Acquisition Risk Management Guide, 
AFMC Pamphlet 63-1 01 , Sept 1 993 
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A group was assembled consisting of people who had been involved in the 
development of several heritage propulsion systems. The group used the criteria 
just discussed to look back in “20-20 hindsight” and assess each component of 
those heritage engines. For each component, the four assessments (one for each 
criteria) were averaged and plotted against the number of rework cycles that had 
actually been experienced. The data were plotted to give the graph shown 
where the Technical Uncertainty Factor (TUF) is the average of the four ratings. 
For each component, there is a clear relationship between TUF and the number 
of rework cycles. Note that the graph also combines the ratings from four 
different propulsion systems. 


Number Of Corrective Actions Correlated With Risk / 
Uncertainty Remaining at Start of Full Scale Testing 
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There are two levers in product development- the efficiency of processes 
and the management of technical uncertainty — that are sufficient to control and 
minimize nonrecurring product development costs. The right quadrant in the 
figure is simply the relationship already shown between the Technical 
Uncertainty Factor and the number of rework cycles (corrective actions). The 
individual points are now covered with a blue colored band. The left quadrant is 
simply the relationship between the number of corrective actions and the total 
cost to perform those corrective actions. The slope of the line is the average 
cost of a corrective action. The horizontal axis labeled Cost to Perform 
Corrective Actions is a measure of the portion of development costs absorbed by 
performing corrective actions. As is apparent from the figure, decreasing the 
Technical Uncertainty Factor before designing the full scale product combined 
with reducing the average cost to perform a corrective action results in a 
dramatic reduction in the cost of the development program. 


Process Improvements and 
Management of Technical Uncertainty 
Sufficient to Cut Non-Recurring Costs 


Process Improvements 



YEARS 


YEARS 
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In the upper part of the figure, a distribution shows the difference between 
using the uncertainty evaluation criteria to evaluate the uncertainty in 20-20 
hindsight (labeled “Actual”) and, for a limited number of cases, the results that 
would have been obtained if the criteria had been used to evaluate technical 
uncertainty at the time the design initiated (labeled “Perceived”). In all cases, the 
technical uncertainty in 20-20 hindsight is higher than would have estimated at 
the time of design. In the middle of the figure, it is shown that a systematic mis- 
estimate in technical uncertainty of 0.2 would lead to a $1 billion difference in 
the size of the development program. The consequences are severe and dictate 
that more accurate techniques be developed. 


Why a Criteria-Based Approach to Assess 
Uncertainty (Risk) is Not Enough 


At time of design, criteria-based assessments are 
optimistic by up to 30 percent of full-scale 



TUF (Actual - Perceived) 


A systematic optimistic assessment of 0.2 (20 
percent of full scale) can lead to $1 billion overrun 


Criteria based approaches are useful at 
system levels but much less useful at level of 
design parameters 
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At the time that the moon landings were coming to an end, NASA awarded a 
contract to go back and look at the development programs for two of the 
primary rocket engines to see what could have been done to make development 
more efficient. One of the results from that study is shown in this slide. There 
are three major fundamental causes of trips through the Test-Fail-Fix cycle. 
First, lack of understanding (or, uncertainty) of the environment. Second, lack 
of understanding (or, uncertainty) of the hardware that was being built. Third, 
limitations in the design process. 


THESE ARE THE FUNDAMENTAL CAUSES. THESE MUST BE 
ADDRESSED IF ONE INTENDS TO MAKE ANY SIGNIFICANT 
DIFFERENCE IN DEVELOPMENT PROGRAMS. RDCS WAS DESIGNED 
TO ENABLE DESIGN TEAMS TO ATTACK THESE CAUSES DURING 
DESIGN. 


Fundamental Causes of Corrective Actions in 
Test-Fail-Fix Cycle (and Operability Cycle?) 


DESIGN PROCESS 
LIMITATIONS (e.g.) - 

• Inadequate state of the art of 
design and analysis 

• Assumption of similarity 
when dissimilarities existed 

• Decision to proceed in face 
of significant uncertainties 



LACK OF 
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OF ENVIRONMENT 
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In this figure, descriptions are provided of the various segments of the pie 
chart from the previous figure. Characterization of the hardware includes 
material properties as well as the manufacturing and quality control processes 
used to convert the raw material form into the actual hardware. Environment 
characterization includes the operation duty cycle and the loads associated with 
each portion of the duty cycle. The portion corresponding to design limitations 
processes the uncertainty information from the hardware characterization and 
from the environment characterization to provide an integrated calculation of 
life, risk, robustness, and reliability. The DARPA-funded Robust Design 
Computational System (RDCS) was designed and developed to provide the 
integrated calculation capability. RDCS is the only system in existence which 
was designed and developed for this purpose, and it is the only system designed 
and developed by engineers who actually work in product design. 
Subsequently, the Accelerated Insertion of Materials program, also DARPA 
funded, developed the capability for materials characterization and linked it to 
RDCS. 


Management of Uncertainty Key to Low Cost Development 


Accelerated Insertion of 
Materials (AIM) 
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The Robust Design Computational System (RDCS) provides three major services to a 
design team: (1) a facility to capture the analysis and design process, (2) options to 
rapidly evaluate the design, and (3) a capability to process large numbers of jobs in 
parallel over a network of workstations or on a high performance computing system. A 
multidisciplinary team must define an integrated set of executable modules (one or many) 
that evaluate the some aspect of the design. This integrated set of modules is linked to 
the RDCS System Director which provides a large number of options to automatically 
create design evaluation instances including deterministic and probabilistic effects. 
RDCS then sends these jobs out to be processed and retrieves and displays the results. 


Robust Design Computational System 

A Domain Independent Comprehensive Tool Set to Analyze the Design Sjpace J 
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The RDCS full factorial design scan feature allows a design team to evaluate 
the design for a systematically generated set of points. In this case a set of 
evaluations was performed for systematic variations in tilt angle, and a second 
set of evaluations was performed for systematic variations in the location of the 
center of gravity. In each case, the selection of design value was based on 
which design values would result in the stresses being well within a low stress 
region. Robustness is achieved because stresses will continue to be low even 
with variations (such as manufacturing variations) in the tilt angle and location 
of the center of gravity. Note that nearly 1000 lengthy nonlinear ANSYS 
solutions were required for this design activity. 


RDCS Design Scan Analysis Identifies Robust Tilt Angle 
and Center of Gravity Location for Minimum Stress 


• Analysis of turbine blade set requires 
162 non-linear ANSYS cases in 2 
overnight runs 

• 6 blade sets have been balanced 
using RDCS for a total of 162*6 = 972 
solutions 


Robust Tilt Angle 
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This and the next two charts illustrates applications of RDCS to actual large 
scale design cases. Specific details have been removed to enable these charts to 
be presented in an open forum. This chart shows the application of RDCS to 
help select appropriate values for the design parameters characterizing the cross 
section of the fuselage of a large aircraft. As in the turbine blade design case, 
factorial design space explorations were combined with sensitivity analyses. 
Determining the appropriate values required 158 large scale finite element 
solutions. Without the ability of RDCS to rapidly generate and process 
evaluation cases, only a few cases would have been run. 


RDCS Application to Fuselage Concept Design 


Application Objective 


* To aid in the selection of appropriate values for 
fuselage and barrel design variables 

* An optimum weight design meeting constraints 
of body bending, ultimate internal pressure, 
decompression and forward and downward “g” 
loads 


High Level Description 


* Design variables: Barrel and frame geometry 
parameters 

* Response Variables: Weight 

* Solvers: RDCS and ANSYS 


Solution Scope 


► RDCS: Sensitivity analysis and Factorial Design 
Space Explorations 

* ANSYS: Static analysis and Optimization 

* Solution Cases: 158 Large Scale FEM Solutions 

— Rocketdy n e — 
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Typical Results 



Internal Pressure 


Decompression 


RDCS Application Benefits 


• Significant insight into the behavior of the 
structure that would otherwise been lacking 

> Rational design decisions 

■ Better and optimal design 

• Automated design process 

• Significant design cycle time reduction even for 
the first application. With RDCS models and 
projects (templates) set up, further similar 
application study can be performed 
automatically in a day or less with engineers 
time totally devoted to design improvement 
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RDCS has also been used to investigate the effect of skin-stringer panel 
geometric parameters on maximum moment at the flange and the margin of 
safety for stringer pull-off. The percent contribution pie chart is one 
representation of the results of a sensitivity analysis, and the pull-off margin of 
safety was evaluated using a factorial design space exploration. For this case, 
81 large scale finite element solutions were required. In the Accelerated 
Insertion of Materials program, a similar problem was analyzed for sheet and 
stringer composed of composite materials. In this case, material processing 
modules were included so that the design variables included, for example, cure 
cycle parameters. Each cure cycle run would take about an hour to run, but 
hundreds of cases could be run overnight due to the parallel processing 
capability of RDCS. 


RDCS Application to Fuselage Concept Design 


Application Objective 


* To aid in the selection of appropriate frame 
spacing and fuselage cross-section parameters 

* Investigate the effect of skin- stringer panel 
geometric parameters on maximum moment at the 
flange and margin of safety for stringer pull-off 


High Level Description 


* Design variables: Skin Thickness, Flange 
Thickness, Stiffner Height, Total Flange Width, 
Cap Width (Hats Only) 

* Response Variables: Maximum Flange Moment, 
Pull-off Margin 

* Solvers: RDCS and ANSYS 


Solution Scope 


► RDCS: Sensitivity analysis, Factorial Design 
Space Explorations 

* ANSYS: Static non-linear large deflection 
analysis 

* Solution Cases: 81 Large Scale FEM Solutions 
— Rocketdyne 



Response Surface: Pull-off Margin of Safety 


RDCS Application Benefits 


1 Rapid factorial design calculations for external ANOVA 
study and response surface with significant cycle time 
reduction 

1 ANOVA helps identify critical factors and interactions 
1 Accurate surrogate response surface model helps 
simplify the design process 
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Turbine power is sensitive to the turbine pressure ratio which is in turn is 
sensitive to turbine nozzle area. Because turbine nozzle area varies with each 
build, analyses were performed to assess the effect of component uncertainties 
on engine performance and to select design parameters such that the 
performance sensitivity is minimized. In this case several components of the 
propulsion system were modeled, variabilities estimated, and 1000 solutions 
were obtained via the Monte Carlo simulation feature available in RDCS. The 
analyses were quickly completed in time to influence a design change. 


RDCS Application to Engine System 


Application Objective 


* Optimize Staged Combustion Cycle Engine to 
meet vehicle requirements 

* Incorporate design solutions to component 
variations for robust engine design 


High Level Description 


* Design variables: Component uncertainties on 
turbopump/turbine efficiencies, turbine nozzle 
flow areas, injector/cooling jacket flow 
resistances, etc. 

* Response Variables: Turbopump speeds and dis 
pressures, PB temp, engine perfo, etc. 

* Solvers: RDCS, SSODO and SSOD 


Solution Scope 


► RDCS: Min/Max Design Condition (MDC) 
Variation Study 

* SSODO: Steady- State On-Design & Optimizer 

* SSOD: Steady-State Off-Design Eng Bal Code 

* Solution Cases: 1000 Solutions 
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The Design Change Based on Robustness 
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RDCS Application Benefits 


• RDCS Provided the necessary variability effect 
to make a design change 

• Incorporated the oxidizer turbine bypass valve 
(OTBV) to minimize MDC 

• Significant design cycle time reduction (approx. 
3 instead of 30 hours of run time) for the 1000 
engine balance predictions. 
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DARPA’s Accelerated Insertion of Materials program is a very well conceived 
program which targets the accelerated development of confidence in hardware 
characterization such that the technical justification is developed for government 
certifying agents to certify that the product can be used. 


Accelerated Insertion of Materials - Composites 


Jointly accomplished by a BOEING Led Team and the U.S. 

Government under the guidance of NAST 

Work funded by DARPA/DSO and administered by NAST through 
TIA N00421 -01 -3-0098 

Dr. Raymond J. Meilunas, Government Agent/Technical Monitor, NAVAIR 
Dr. Leo Christodoulou, DARPA/DSO Program Manager 
Dr. Steve Wax, Director, DARPA DSO 



NAV^A I R 



Rocketdyne 
Propulsion & Power 


QjriF&ArB 1 9 


49 



There are two major elements of the Accelerated Insertion of Materials 
program which result in accelerating the development of confidence in a new 
material to the point where it can be inserted into a design. First is a tool set 
which integrates physics-based models, helps focus testing to validate models 
and fill in the gaps in knowledge, and then fuses the analytical and test 
information to produce mathematically defendable statements of confidence. 
The second element is a methodology which guides the use of the tools, 
provides a framework for the interaction of the various disciplines involved in 
the design, and ensures a broad consideration of all factors which might become 
“show stoppers.” RDCS is used as the framework for integration and 
computation. 


Accelerated Insertion of Materials -- Composites 

(AIM-C) 
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At the top level, the methodology looks like several TRL (Technology 
Readiness Level) scales each for a different category of “Show Stopper.” The 
categories of show stoppers were developed from a brainstorming session with 
materials development experts who listed all the factors that could potentially be 
showstoppers in inserting materials. The factors were then affmitized into the 
categories shown, and top-level TRL criteria were developed for the various 
levels. 


For Each Category, Path to Certification Consists of 
Product Readiness Levels (Steps) With Exit Criteria (Gates) 
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AIM-C Technology Readiness Summary 
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The matrix described in the previous chart is the mechanism which ensures 
that all factors that might impede insertion of the material are addressed in a 
timely manner. The chart gives examples of the kinds of issues that are 
addressed in several categories at selected TRL levels. 


Technology Readiness Levels 

Defining all the Questions and Measuring Progress 

NAV^A I R 



^ Concept Definition 

• Application Definition, Loads, Environment 

• Concept Refinement 

Concept Definition/ Applications Revised 
By Lamina Data 

• Design/Geometry Parameter Studies 
Heuristics, Simulation, Test 

Assembly Concept 

• Effect of Assembly on Performance 

• Effects of Defects 

Production Materials/Materials Specs 
• Material Property Studies 


Cost Benefit Elements ID’d 
And Projected 
• Performance Data for Trades 

Rocketdyne 
Propulsion & Power 
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Target Application Pilot Production 
of Generic full Size Parts 
• Effect of cure/tooling on Performance 


Preliminary Properties- 
Characteristics 

• Analysis/Test-Generated Design Values 

• Effects of variability 
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What is unique about the Accelerated Insertion of Materials program, 
however, is that a series of “xRLs” have been defined beneath each of the TRL 
categories shown on the matrix. These xRLs are basically a decomposition of 
the top level criteria down to associated criteria for each individual discipline 
involved in design. At the very bottom are exit criteria which must be met to 
move to the next TRL level. When these exit criteria are quantitative, then the 
qualitative TRL criteria at the top level have been converted to quantitative 
criteria. The conversion of qualitative criteria to quantitative criteria is 
apparently unique to the Accelerated Insertion of Materials program. The 
combination of physics-based models and focused testing is then used to 
establish that the exit criteria have been met. 


AIM Methodology Becomes a Requirements Flow 
Down, Exit Criteria, and a Completion Roll Up 

nav^air 
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Exit Criteria 
For xRLs 


The Same Linkage Used 
To Flow Down Requirements 
Is Used to Roll Up Knowledge 
And Track Progress as 
Designer Knowledge is 
Gathered. 
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Use of Prior Knowledge 
Recommended Analyses 
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Recommended Combination of 
Prior Knowledge / Analysis / Test 
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This chart gives a sense of how many physics-based models actually get 
involved in establishing confidence in a material property. The effects of 
variability are included via the RDCS capability to perform probabilistic 
analyses using models which were developed deterministically. 


Quantifying Uncertainty 

Robustness to Flaws, Geometric and Material Variability 

Probabilistic Analysis - RDCS Math Model 

N AV^A I R 
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Similarly to the case of selecting design values for the turbine blade discussed 
earlier, this chart is the result of a design space scan to determine parameter values 
(Lstiff and Lower radius) which minimize the energy available to propagate a 
crack. What is different here is that underlying the results are physics-based 
models for the constituents and the processing of composite materials along with 
the geometric parameters associated with the structure itself. 




Minimizing Uncertainty by Design 

NAV^A I R 

Geometry Effect on Radius and Edge-of-Flange Delaminations 


Large Lstiff and 
smaller lower radius 
minimizes energy 
available to 
propagate a crack 

SERR in- lbs in^2 


Red: G, EOF 

Blue: Gj Lower Radius 

Green: G n /4 EOF 


Brown: G n /4 Lower Radius 
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This chart shows the results of using a Monte Carlo approach to predict the 
distribution of failure load for a composite structure given uncertainties in the 
constituent properties, processing parameters, and geometric parameters. 
Although the cure cycle simulations could take up to an hour to run, the parallel 
processing capability of RDCS was able to process 1000 instances overnight. 


0DARPA' 


Handling Uncertainty - The AIM-C Approach 
Quantifying Uncertainty 

Example - OHT Laminate Monte-Carlo Simulation 
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Over the last 14 years, studies of aerospace development programs have 
identified a Product Development Imperative, management of uncertainty, and 
the dominant drivers of uncertainty. DARPA investments in the Robust Design 
Computational System and in the Accelerated Insertion of Materials program 
have matured the relevant technologies sufficiently that it is now feasible to 
demonstrate the technologies in a controlled pilot project in parallel to an 
ongoing program. 


Summary 



1. Product Development Imperative - Developing advanced technology products on 
budget and schedule requires effective management of uncertainty 

• Classical risk management practices are inadequate 

2. Uncertainty the primary driver in program cost and schedule overruns 

• Hardware characterization 

• Environment characterization 

• Design process limitations 

3. RDCS designed to eliminate design process limitations driver by providing 
framework for quantitative assessment and management of uncertainty 

4. AIM technologies accelerating technology insertion (attacking hardware 
characterization driver) by linking quantitative characterization of materials and 
processes with RDCS 

5. AIM technology sufficiently mature to 

• Generalize to include environment characterization 

• Evaluate in controlled pilot project working in parallel to ongoing program 
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Physics-based Conceptual Design of Revolutionary Concepts: 
A “Paradigm Shift” in Complex System Design 


Dimitri Mavris 

Aerospace Systems Design Laboratory 
Georgia Institute of Technology 
Atlanta, GA 
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Dr. Dimitri Mavris is an Associate Professor in the School of Aerospace Engineering 
at Georgia Tech. He is also the director of the Aerospace Systems Design Laboratory: the 
largest research lab in the School of Aerospace Engineering. Dr. Mavris advises 128 
masters’ and Ph.D. students engaged in relevant research in the field of complex systems 
design. His current research focus includes efforts in the design of unconventional 
systems and high-fidelity physics-based design methodologies. 
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The main motivation behind a physics-based conceptual design 
approach is the focus on unconventional systems for which no “canned” 
design programs exist. Currently, NASA has identified several classes of 
unconventional systems that it wishes to examine over the next several 
years. Some of those configurations are highlighted here. Unfortunately, 
since these systems are extremely unconventional, reliance upon 
historical data is often inappropriate. As such, efforts are underway at 
Georgia Tech to more fully understand these systems and model them in 
a variable-fidelity physics-based design environment. 


By physics-based, we mean that aircraft drag polars will be more 
accurately calculated using a panel method or CFD code. The propulsion 
systems will be analyzed from a cycle standpoint for both design and off 
design operation. The performance of the aircraft over many flight 
regimes will likely involve an energy or exergy-based approach to 
tracking the various performance constraints imposed upon the vehicle 
by the mission requirements. Structural analysis will require some sort of 
higher-level modeling than traditional zeroth order design codes that 
focus only on historical mass estimation relationships. 


Motivation for Physics-based Conceptual Design 


Subsonic Transports 


Supersonic Aircraft 


Roto rc raft 



Uninhabited Air Vehicles 


Personal Air Vehicles 


New Generation of Vehicles can 
not be modeled accurately in the 
absence of historical data 


Extreme STOL 
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This chart shows the paradigm shift in design that must occur. This is 
the pre-2001 acquisition timeline for military acquisition programs. The 
milestones have shifted slightly but the basic concepts are the same. This 
chart indicates the “today” state of affairs as a dashed line and the future 
goals as a solid line. Essentially, it has been discovered that a majority of 
the cost of the program is committed at the early stage of the design 
process, when the knowledge about the design process is very low. 
Furthermore, early in the design process, since little is fixed, the design 
freedom is much greater than after the configuration has been specified 
and heavily analyzed. 


Through the infusion of physics-based conceptual design methods, 
more information about the design (knowledge) is moved forward in the 
design process. As a result, design freedom increases because more 
designs can be examined in the conceptual design phase. As a result, the 
cost committed curve shifts to the right, because major design decisions 
do not fix the design early in the process due to the increase in freedom. 


Physics-based Conceptual Design - A Paradigm Shift 


Acquisition Timeline 



Requirements 

Definition 
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Design 
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Design 
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A traditional point-design philosophy is a deterministic analysis that is 
usually driven by historical data and disciplinary-centric design 
organizations. In these designs, almost all the requirements and 
assumptions are fixed. The process is also very time consuming and 
often involves a manual passing of information from designer to 
designer in a “throw-it-over-the-wall” type approach. 


The ASDL approach seeks to reduce cycle design time to allow 
existing organizations to be more effective. Several enabling 
technologies are required to make this transition possible. 


Traditional, Point-Design Philosophy 


• May be characterized as a manual, deterministic, data driven, serial or 
parallel, disciplinary-centric, point design process 

• Design requirements, and technology assumptions are usually fixed 
and a design space exploration is performed around one or a handful 
of concepts (point solutions) 

• As organizations strive to decrease costs and reduce operational 
overhead, the number of personnel available for given activities is 
decreasing 

• At the same time, the demands on the organization for more in depth 
analysis at the conceptual and preliminary stages is increasing 


• Asa result, a paradigm shift is required to reduce design cycle time, 
allow for more iterations, and increase fidelity 


• Traditional organizations can be supported and enhanced by several 
enabling technologies, to be presented here, that allow for this 
transformation to take place in a practical fashion 
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To facilitate this transition, a critical element is an integrated 
modeling and simulation environment. The automation of the design 
process using a commercially available tool will allow the ability to 
perform physics-based design without the reliance on historical data. 
This environment will also allow the creation of parametric tradeoff 
environments, in which a dynamic design space can be created. 


Using this environment, probabilistics can also be brought into the 
design process to quantify risk. 


A key enabler for more advanced design using integrated systems is 
intensive computer power. 


What is needed for the Paradigm Shift to occur? 


• Transition from single-discipline to multi-disciplinary analysis, 
design and optimization 

• Automation of the resultant integrated design process 

• Transition from a reliance on historical data to physics-based 
formulations, especially true for unconventional concepts 

• Means to perform requirements exploration, technology infusion 
trade-offs and concept down selections during the early design 
phases (conceptual design) using physics-based methods 

• Methods which will allow us to move from deterministic, serial, 
single-point designs to dynamic parametric trade environments 

• Incorporation of probabilistic methods to quantify, assess risk 

• Transition from single-objective to multi-objective optimization 

• Need to speed up computation to allow for the inclusion of 
variable fidelity tools so as to improve accuracy 
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Furthermore, uncertainty is prevalent at every step in the design 
process. By examining the disciplinary uncertainty for each of the design 
tools, trades and “what if’ scenarios can be performed to make the 
designer aware of this inherent uncertainty in his or her decisions. 


The latest advances in integration technology allow for a 
collaborative design to be performed across multiple departments, and in 
some cases, even across multiple geographic locations. 


Replacing higher-fidelity tools with surrogate models (metamodels or 
Response Surface Equations) allows information to be brought forward 
in the design process. These methods allow the integration of legacy 
tools, and allow the user to see what is happening inside the tools by 
examining trends and validating whether these trends represent the 
physics of the problem. This advantage is called “transparency.” 


Finally, multi-attribute decision making techniques can be used to 
account for the fact that objective functions seldom contain only one 
objective. Decision making can be facilitated by using collaborative 
tools to quantify customer requirements when possible. 


Elements needed to enable this Paradigm Shift 


Advances in MDA/MDO methods and techniques to encompass the 
holistic nature of the problem, emphasis on uncertainty associated with 
the early design phases 

Creation of computational architecture frameworks to allow for easy 
integration and automation of sometimes organizationally dispersed tools 
Emergence of commercially available frameworks will further expedite 
the usefulness of the proposed approaches 

Creation of physics-based approximation models (surrogate or meta- 
models) to replace the higher fidelity tools which are usually described as 
too slow for use in the design process, cryptic in their use of inputs, 
interfaces and logic, and non-transparent (lack of proper documentation, 
legacy) 

Use of probability theory in conjunction with these meta-models will 
enable us to quantify, assess risk and to explore huge combinatorial spaces 
In fact it will enable us to uncover trends, solutions never before examined 
in a very transparent, visual, interactive manner 
Use of Multi-attribute decision making techniques, pareto optimality, 
genetic algorithms to account for multiple, conflicting objectives and for 
discrete settings 

?S"S'‘^ Dr Georgia Institute of Technology 
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Surrogate models are a key enabling technology for physics-based 
design. The first and primary benefit of these models is the acceleration 
of the design process. An additional benefit is that a surrogate model can 
be created around a proprietary preliminary design tool. This model 
cannot be reengineered to produce the tool or understand what is 
happening inside it. The model merely replaces the proprietary tool with 
a rapid black box that is only defined for a range of inputs for the 
problem that is specified. The mother organization retains the ability to 
use the original code and can make surrogate models for a variety of 
problems, controlling their access. 


These models can be used to rapidly trade off requirements, 
technologies, and design concepts during the early design phases. 


Key Enabler - Surrogate Models 

Reliance on meta-models or surrogate models as a means to : 

• Speed up processes, protect proprietary nature of codes used, 
overcome organizational barriers (protectionism of tools and 
data), allow for the framework to be tool independent (no need 
for direct integrations of codes), this also enables our desire for 
variable tool fidelity formulations, further it will allow the 
designer to perform requirements exploration, technology 
infusion trade-offs and concept down selections during the early 
design phases (conceptual design) using physics-based methods 

• These surrogate models can also be used at the integrated system 
level to determine responses at that level. This will allow us to 
move from deterministic, serial, single-point designs to dynamic 
parametric trade environments 
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This slide highlights some of the enabling tools and techniques in use 
at the Aerospace Systems Design Lab. Several of the tools have been 
borrowed from other disciplines and modified to suit the systems design 
problem. ASDL students and researchers have additionally developed 
the methods at the bottom of the page for specific, higher-level 
applications. These methods generally indicate structured approaches to 
problem solving, technology identification, and robust design. 


Enabling Tools and Techniques 
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Established Techniques 

Response Surface Method (Biology; Ops Research) 

Design of Experiments (Agriculture, Manuf.) 

Quality Function Deployment, Pugh Diagram (Automotive) 
Morphological Matrix (Forecasting) 

MADM techniques (U.S Army, DoD) 

• Uncertainty/Risk Analysis (Control Theory; Finance) 

ASDL Innovation 

Feasibility/Viability Identification 
Robust Design Simulation (RDS) 

Technology Identification, Evaluation, Selection (TIES) 
Joint Probabilistic Decision Making (JPDM) 

Unified Trade-off Environment (UTE) 

Virtual Integrated Stochastic System Technology 
Assessment (VISSTA) 
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As our colleagues from the PIDO community have mentioned, there 
is a tremendous advantage to linking design codes in an automated 
environment. Typically, a user expects to see at least an order of 
magnitude reduction in design cycle time once a suite of tools is 
integrated. 


Integrated Design: Reduction in Cycle Time Through Automation 


• Performing an integrated design involves linking conceptual and 
preliminary design tools in a computational environment that 
automatically passes information between design codes 

• Enablers: 

- Computational environments such as ModelCenter or iSIGHT, . . . 

- Design codes with simple inputs/outputs without hard coding of design 
variables or internal optimizations that may skew results 

• Integrated design provides tremendous advantages in design 
cycle time by eliminating the re-keying of inf ormation from 
output files to input files. 

• For example, a missile design environment was programmed as 
an integrated suite of codes. It takes 35 seconds to perform a 
design. If the codes were not linked, it would take approximately 
45 minutes to pass the information back and forth and check for 
errors! 
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A key feature of the ASDL approach is the fact that often, codes of 
varying levels of fidelity are required at different stages of the design 
process. Shown here is a sizing and synthesis-centric approach to design 
whereas the geometry and mission analysis rely on the disciplines in the 
two outer circles. For the conceptual design phase, the inner circle is 
used. These methods traditionally involve table lookup routines, 
response surface models, and other surrogate models for rapid design 
space exploration. 


The outer ring represents the preliminary design tools, that are 
traditionally more accurate and require more computing resources to run. 
These analysis can either be approximated with surrogate models or 
directly linked to the analysis. Obviously, the first choice is desired for 
expediency, and the second choice for accuracy. A combination of the 
two are utilized in a variable fidelity modeling and simulation 
environment. 


Varying Fidelity M&S Initiative 
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The key to the physics-based modeling and simulation environment is 
the robust design simulation, which is essentially the linked process 
using an automation tool. This environment is subject to design 
constraints such as requirements, and environmental constraints that are 
usually very stringent. Furthermore, uncertainty can be brought in to this 
process, and a gap analysis can also be performed to determine which 
technologies are required to make a system feasible and viable. 
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Whereas the integrated design environment can be used to run a 
single point in a deterministic way, the key advantage to this 
environment is its ability to take in distributions of inputs to both explore 
the design space and quantify uncertainty. 


Parametric Design: Using an Integrated Design on a Large-Scale 


• The integrated design environment is an enabler for a parametric 
design study 

• Instead of passing in a series of input variables, a parametric 
design can take a distribution of inputs. 

• In this manner, an entire design space can be explored, rather 
than small perturbations around a single point design 

• Large design spaces may take too long to explore by traditional 
means 

- The integrated design environment above can be used to generate 
metamodels of the design process 

- These metamodels, custom made for a given range of inputs, can be 
evaluated in a spreadsheet hundreds of times per second 

- Metamodels represent another order of magnitude in reduction for design 
cycle time. 
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The following several charts show a variety of examples and 
how the necessary pieces come together. Shown here is a mission 
profile for a High-Speed Civil Transport (HSCT) aircraft. The 
requirements are set by the customer, and are typically fixed. 
ASDL’s requirements exploration allow these requirements to 
become variables in the process, so that the “showstopper” 
requirements can be quickly identified and perhaps relaxed if they 
are merely “desirements.” 
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With the requirements defined, a potential class of vehicles must be 
defined. A structured means of doing so is with a Morphological Matrix. The 
morphological matrix is nothing more than a decomposition of all possible 
contributing elements of the system. It is a means to brainstorm and think out 
of the box for potential solutions to the problem. 


For example, the project manager could bring together all of his experts 
and decompose the system. Do we want a wing and tail vehicle? Or a wing 
and canard? And so on. If you do this for each element of the system, then 
you have effectively defined the alternative concept space which may have 
mission parameters, technologies, and so on. 


Once this matrix is sufficiently defined, one must establish a baseline to 
continue on with the TIES method. You do this by selecting one element from 
each row like the circled items, usually present day capabilities. This is your 
baseline that you will do all deviations on. 


Next, that system is further decomposed into geometric and propulsive 
parameters that will define the design space to be investigated for feasibility. 



Define Concept Space: 

Morphological Matrix 


• Purpose: Establish the concept space that may fulfill the customer requirements and establish a 
datum point for the feasibility investigation 

• Performed with the aid of the Morphological Matrix technique 


• Procedure: 

- Define Alternatives Space 

• Functionally decompose the existing 
system into contributing 
characteristics 

• For each characteristic, list all the 
possible ways in which it might be 
satisfied 

• Select a datum point; permutations 
are concept alternatives 

- Define Design Space 

• Further decompose the system from 
the Alternatives Space to elementary 
attributes, such as geometric and 
propulsive characteristics 
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This is an example of a series of outputs generated based on a “man 
in the loop genetic algorithm” for a supersonic business jet design. Each 
of these configurations represents an aircraft that has had an 
aerodynamic analysis to calculate accurate drag polars and a complete 
propulsion cycle analysis. A parametric engine deck has been generated 
for each of the configurations and they are all fuel balanced for the same 
mission. 


The creation of one of these cases can take up to a day with no 
surrogate models or under a minute if effective surrogate models are 
used. The man in the loop genetic algorithm allows the designer to view 
certain configurations and to highlight those which he or she thinks are 
infeasible based upon designer intuition. Often, characteristics like 
flutter and divergence are not analyzed in the conceptual design phase; 
however, a trained engineer can determine which configurations are 
undesirable from that approach. This design method combines the 
advantages of rapid run time with surrogate models with higher fidelity 
analysis AND designer intuition. 


Example of a Parametric Design Exercise for a Supersonic Business Jet 
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Each aircraft to the left is an 
example of a complete design. 
Parametric design provides the 
user with the power to test 
hundreds or thousands of designs, 
where previously, time permitted 
a single design point only. 

Each aircraft to the left has 

- A complete analysis of the 
propulsion system 

- An aerodynamic analysis to 
calculate accurate drag polars 

They have all been sized for the 
mission requirements, which are 
ALSO parametrically scalable. A 
change in desired range will re- 
generate this matrix of designs. 
The creation of a single one of 
these aircraft designs can take less 
than a minute or up to a day , 
depending on the desired fidelity 
of the design tools. 
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Additionally for selected configurations, the designer can request a 
sonic boom analysis both to narrow the field of candidates and to 
improve his/her expert intuition. Shown here is the blue baseline 
configuration. The sonic boom overpressure is decreased in the red 
configuration due to the increased sweep of the aircraft. The boom 
problem is essentially solved by the third configuration; however, issues 
such as takeoff rotation, flutter, and the construction of this highly swept 
configuration clearly pose issues to the designer. The green 
configuration indicates a solution proposed by the genetic algorithm, but 
clearly the sonic boom is larger than the baseline case. In this manner, 
different configurations can be examined in real-time. The computer 
program likes the third choice, but the experienced designer does not. 


Man in the loop Genetic Algorithm - 
Sonic Boom Profiles for Various SBJ Configurations 





Conventional Baseline 
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Now that you have a general concept baseline definition, you 
must establish the design space for which you will investigate 
feasibility and viability with respect to the customer requirements. 
In this example, both geometric and engine cycle parameters are 
considered, such as wing area, fan pressure ratio, and planform 
geometry definitions. 


Note: The geometric and 
propulsive parameters may 
vary in the ranges defined 
with the same likelihood 
since at the outset, there 
should be no preference of 
values. Hence, uniform 
distributions are assigned 
to each parameter. 



Define Concept Space: 

Define Design Space 


Variable 

Minimum 

Maximum 

Units 

Description 

SW 

7500 

9000 

ft 2 

Wing area 

TWR 

0.29 

0.33 

~ 

Thrust-to-weight ratio 

TIT 

3000 

3400 

°R 

Turbine Inlet Temperature 

FPR 

3.5 

4.5 

~ 

Fan Pressure Ratio 

OPR 

18 

21 

f 

Overall Pressure Ratio 

CLdes 

0.08 

0.12 

~ 

Design lift coefficient 

X2 

1.54 

1.69 

~ 

LE kink x-location* 

X3 

2.1 

2.36 


LE tip x-location* 

X4 

2.4 

2.58 

~ 

TE tip x-location* 

X5 

2.19 

2.37 


TE kink x-location* 

X6 

2.18 

2.5 

~ 

TE root x-location* 

Y2 

0.44 

0.58 

~ 

LE kink y-location* 

t/c_root 

3 

5 

% 

Wing root t/c ratio 

t/c_tip 

2 

4 

% 

Wing tip t/c ratio 

SHref 

400 

700 

ft 2 

Horizontal Tail area 

SVref 

350 

550 

ft 2 

Vertical Tail area 


* Variables Nondimensionalized by wing semi-span 
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From the previous slide, here are the possible wing planforms for the 
HSCT. All these planforms at least appear feasible to the designer and 
can be carried on for further analysis. 


Possible Wing Planforms 
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Graphically, a modeling and simulation environment looks as shown, 
whereas the engine performance analysis and weight estimation are 
coupled with the aircraft design and life cycle cost code. Emissions, 
economics, noise, required thrust, and vehicle performance are all 
analyzed in this environment. 


Creation of Modeling and Simulation Environment 



f Fidelity 
l Multipliers . 


Economic 

( Market 

.Assumptions 


zsl Requirements. 


Mission 

Requirements 


Multi-Disciplinary DOE 




Technology 

Setting 


Airframe Fixed! 


NEPP 

Engine Performance 
Program 


r Given Engine Architecture 


FLOPS 

Flight Optimization 
Code 


Vehicle Size 
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A dynamic tradeoff space for an HSCT is shown here. The various 
outputs are on the y-axis and the various inputs are shown on the x-axis. 
This environment can be operated like a calculator, where the red 
hairlines on the x-axis can be set to different values between the low and 
high range for each variable. When these values are changed, the slopes 
for the entire calculator update due to multivariable interactions. The 
slopes of the lines indicate the partial derivative with respect to each of 
the X’s with all other variables held constant (for example, the upper left 
box is the partial derivative of takeoff gross weight with respect to wing 
area, and so forth). 


These lines are also an indicator of the required fidelity of the 
analysis codes. If a slope is steep for an analysis that has a low fidelity, 
then the penalty for missing the correct value is amplified because it has 
a larger impact on the candidate response. 
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Shown here is a surrogate model integrated into a dynamic requirements exploration 
space. This example is for an F-18E/F fighter, where the parametric space for the F- 
18C/D was to be stretched to include the E/F derivative. The white area in the above plot 
indicates a feasible design region. The hairlines are set to the design point for the F-18C. 
Note: the information on this chart has been altered to preserve any sensitive data values 
and cannot be used to back-out any proprietary information. 


Notional Parametric Dynamic Requirements Exploration 
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When the combat radius slide bar is increased (meaning more combat radius is 
desired), the design space shrinks. As the hairlines indicate, the F-18C now has 
insufficient excess power to accomplish this mission. 


Notional Parametric Dynamic Requirements Exploration 


As the combat 
radius slide bar 
is moved to the 
right, the design 
space shrinks to 
reflect the new 
constraint. 
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With a parametric design, 
the slide bars to the left 
are now enabled to give 
the designer fr eedom to 
change the space, moving 
both the design point 
AND the constraints 
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The design space continues to shrink as the combat radius is decreased. 


Notional Parametric Dynamic Requirements Exploration 
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A new environment was created with different contours when it was discovered that 
the previous environment was not entirely accurate. The engine deck provided on the previ- 
ous chart was for an F-18C that uses the F404 engine. The F414 was produced for the F- 
18E/F, and the F414 engine has a better SFC (specific fuel consumption) than its predeces- 
sor! 


As a result, a slide bar for the SFC was placed into the environment. When this slide 
bar is set to the value of the SFC for the F414, a region of the design space opens up. The 
hairline values (sanitized to preserve proprietary information) are set to the F-18E/F, 
which falls in the feasible region. 
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The surrogate models (response surface equations) can also take 
inputs in the form of different distributions. These distributions are 
actually thousands of discrete runs that are rapidly executed in the 
environment. The parameters of the input distribution are defined, and 
the cases are run, producing a cumulative distribution function (CDF) of 
the output on which the probability of success of meeting various goals 
can be established. 


Without surrogate models, this analysis would take too long to be 
useful to a designer. 


Probability Distributions Can Be Input Into the RSE’s 



• With probability distributions, thousands 
of designs across a user-specified 
distribution can be analyzed 

• This allows a designer to assess technical 
feasibility and economic viability 

• Without RSE’s or metamodels, this 
analysis would be impossible, due to the 
execution time of large parametric spaces 

• RSE’s are an enabler for this method of 
design 
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A sample of a probabilistic output for four goals is shown here. This 
aircraft has a low but finite amount of the design space that meets the 
approach speed and flyover noise constraints. Note, if the probability of 
success is too high, the requirements for the design are likely too loose. 


However, also of note is the two showstopper cases in which none of 
the design space can satisfy the requirements for sideline noise and 
average required yield/passenger mile. 


These two failures indicate that technologies must be infused to 
reduce noise and increase economic viability. ASDL methods allow the 
quantification of how much improvement is needed, and what the 
probability of success in the presence of uncertainty and noise will be. 
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By looking at a contour plot of the Joint Probability Distribution and 
overlaying the future target values, the combination of technology 
metrics that will achieve the target can be determined. The contours 
represent lines of isoprobability. The skew in the distribution means that 
the two metrics are correlated. 


Visualizing Potential Solutions to Meet Future Goals 


• Assuming the Goal distributions are 
normal, plot Joint distribution and 
overlay future Goal target values to 
determine if any combination of 
technology metrics will achieve the 
target 

• Interpret the j oint probability as : 

- Highest frequency implies that of the 
combinations considered, the majority 
of the solutions will fall in this region 

- The outer edges or rim represent the 
limits of what can be obtained with the 
technology metric ranges that were 
specified 


Rim of lowest 
frequency of 
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The data points from the Monte Carlo Simulation can be placed onto 
the contour plot, and the data points that meet the goals can be extracted. 
When multiple combinations exist that satisfy the new goals, then the 
selection of the appropriate path may be determined by inspection. For 
example, looking at the path of least resistance, looking at a physically 
realizable solution, or balance with other metrics (noise, cost, vehicle 
performance, or development time). 


Selecting Potential Solutions to Meet Future Goals 


To acquire the technology metric values 
that meet the new Goal values, simply 
overlay the Monte Carlo Simulation 
data 

Pick the data points that meet the goals 
and extract the associated technology 
metric values 

If multiple combinations exist that will 
satisfy the new goals, then selection of 
the appropriate path may be determined 
by: 

- Path of least resistance (based on the 
degree of difficulty) 

- Physically realizable solution 

- Balance with the impact to other metrics 
(noise, cost, vehicle performance) 

- Balance with the cost to achieve and the 
time to develop with the performance 
capability needed 



will satisfy both goals 
concurrently. 
Extract and investigate 
needed capability 
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These joint probability distributions can be viewed in multiple 
dimensions. If the C02 vs NOx plot on the left is used to examine 
whether configurations meet the emissions goals of the program, the 
highlighted cases in the green region can be changed to blue X’s. These 
values then appear instantly in the other plots shown on the right (using a 
software tool called JMP Statistical Discovery Package). As a result, the 
designer can instantly see whether her or she likes the selected points, 
and whether those points meet the noise and economic goals of the 
program. 


From this analysis, some of the selected points meet both noise goals. 
All selected points meet the economic goal (% DOC+I) but none of the 
highlighted points meet the unbumed hydrocarbons (HC) goal. This 
requirement must be relaxed or technologies must be infused. 


Joint Probability Distributions in Multiple Dimensions 


g 


o 



Points that satisfy emission reduction 
goals for C0 2 and HjQ K can be queried 
in the other dimensions 
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The technology forecasting environment (TIF) previously shown has 
another additional benefit. A “gap analysis” can be performed using this 
environment to see the relative improvement required in the various 
design parameters to meet the program goals. These settings can 
determine what types of technologies may be required. 


For example, if a reduction in wing weight and fuselage weight is 
required to meet economic goals (due to a lighter aircraft burning less 
fuel and hence costing less to operate), then advanced structural material 
technologies may be required to meet the goals of the program. 


Technology Impact Forecast Environment 
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Environment 

• Identify code fidelity needed 
to model a technology 

• Impact of degradation of a 
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• This environment provides 
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these trends was invisible to 
us before the parametric 
environment was in place. 
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Weight 
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Engine 

Weight 

° FS 0 % 

Noise 

Suppression 

° 3 0% 

Supersonic 
l Drag 
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Finally, this environment can also be used with TECHNOLOGIES across the x- 
axis instead of design variables. In this case, the technologies have two settings, off 
and on. Turning a technology on as shown initiates a step-change in the output 
responses. In this example for a large passenger transport, both engine and airframe 
technologies were examined in conjunction. Adding technologies reduces or 
increases the responses on the left, and also re-evaluates the engine flowpath code 
to produce a new picture at the bottom left of the screen. This is helpful for our 
collaborative partners from the engine community, who can look at this flowpath 
and determine whether the highlighted configuration is feasible based on their 
engineering intution. 


Notional Dynamic Technology Environment 
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As technologies are added, their combined 
effect on the emissions goals can be quickly 
evaluated. This is shown for the combination of 
technologies T1 & T2 below. 

Baseline +T1 +T2 +T3 +T4 +T5 +T6 +T7 +T8 +T9 +T10 +T11 



1 

HBH ^ 

m 

\ 

— 

luction in C02 


-5.25' 

■4 




-50% 

1 | 



— ■ 8 



Georgia Institute of Technology 
Aerospace Systems Design Luboratoiy 


91 



Project PROMETHEUS is an ASDL-inspired initiative to bring 
higher fidelity analysis to the hands of the decision maker and the 
conceptual designer through the use of advanced design methods and 
high power computing systems. 



Project PROMETHEUS 
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PROMETHEUS will examine using advanced design methods on 
unconventional problems that require collaboration and physics-based 
design. 


Our Motivation 


• High-fidelity, physics-based analyses need inclusion 
earlier in the Design Process 

- Advanced Concepts 

- Multidisciplinary Design 

- Complex Tradeoffs 

- Shortened Design Cycle 

- Et cetera 

• Low-order results not trustworthy to guide vehicle 
definition outside results of historical database 

• Utilization of CFD, FEM, . . . during the conceptual design 
phase is a figurative Holy Grail 
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The PROMETHEUS endeavor is being applied to six graduate design 
competitions this year, as shown above. These six competitions span a 
wide range of designs. 


Also, all of the designs in this year’s competition feature multi- 
mission capability. From cruise missiles that can loiter and dash to 
tunable infrared signature missiles to intelligent UAV’s, the 
PROMETHEUS project is looking at unconventional systems that also 
have the versatility and robustness to perform several missions. 


Varying Fidelity M&S Initiative 


Parametric 

Robust 

Optimization 

Multidisciplinary 

Environment for 

Technology and 

Hyperspace 

Exploration of 

Unconventional 

Systems 



Access to Space (TBCC) 
Hypersonic Missiles 
Strategic Missiles 

Supersonic Vehicles 
Morphing Vehicles 


Unmanned Air Vehicles 
Electric Propulsion 

(Fuel Cells, Solar, Hybrids) 
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As the result of an Office of Naval Research (ONR) Defense 
University Research Instrumentation Program (DURIP) grant, the ASDL 
has constructed a nearly ~$1M facility on the Georgia Tech campus to 
facilitate advanced collaborative design and decision making, as well as 
serve as a central processing area for distributed computing. The 
PROMETHEUS teams are actively using the CoVE for their design 
competitions this year. 


Collaborative Visualization Environment - CoVE 


• The CoVE is a large projection screen permitting multiple 
linked design applications to be simultaneously displayed. 

- Will synchronize early conceptual design tools with high- 
fidelity analysis programs. 

- Key decision-makers will make design choices on-the-fly 
and will immediately see the impact of their decisions. 

• High fidelity tools will be run via a Beowolf cluster: 

- Will provide high computational power. 

- Parallel computing will run design applications 
simultaneously. 

• Backup Storage 

- Several terabytes of memory dedicated to each project. 

- Will permit CoVE users to access previous design iterations. 
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Real time analysis of problems — With the near real time computing 
capabilities, analysis and design can be accomplished by a team in a 
revolutionary time frame. The design environment at ASDL will 
transform how engineering design is approached in all fields of 
engineering. Physics based computing and real time analysis can be 
used in industries ranging from aircraft design to submarine design. 
This is the wave of the future. 

Ability to design with team — In this dynamic environment with the 
near real time capabilities, all the key decision makers will be able to 
actively participate in the decision making and design process of the 
different functions of the CoVE. This will allow for designs to be 
created in significantly less time and with less hassle of correspondence. 

Integrated design tools — Using system integration method software, 
such as “Model Center” or “Fiper”, to coordinate the response of several 
design software tools decreases the design time by eliminating the setup 
procedures associated with each individual procedure. 

State of the art visualization — The state of the art display screens 
enable decision makers to view the results from analysis in plain view. 
The hassle of using multiple computers and multiple viewing windows is 
eliminated. With the simulation and design results in one viewing area, 
the design team can discuss the results devoid of clumsiness associated 
with using multiple standard computer monitors. This state of the art 
visualization will also enable the designers to add enjoyment to their 
tasks by offering a “sci-fi”, or movie like environment to work with. 
This added excitement will ensure that the designers and engineers using 
this environment will become more productive and efficient. 


The CoVE Vision 


• Collaborative Design Environment 
with Advanced Visualization 

• High-fidelity Conceptual Design Tool 
at near real-time results 

• Real-time analysis of problems 

• Ability to design collaboratively with 
team locally or across geographically 
dispersed locations 

• Integrated design tools 

• State of the art visualization 

• Multi-disciplinary optimization in a 
collaborative environment 

• Physics based computing 

• Integrates decision makers to the 
design process 




distributed collaboration 


ASDL CoVE- March 2004 
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Multi-disciplinary optimization in a collaborative 
environment — Any design is created by a design team. In several 
cases, design teams consist of individuals with conflicting schedules or 
physical barriers. Until now, design time has spanned months or years 
according to varying complexities because of three reasons; long 
analysis using many programs, time availability and long distance 
distribution of key decision makers. The CoVE environment will 
drastically decrease the amount of time associated with a single design 
because of its multi-disciplinary optimization and collaborative 
environment. With this state of the art design environment, physics 
based computing and simulations can be run and displayed in near real 
time allowing for key decision makers to see results and make decisions 
in one conference. Decision makers from afar can participate with the 
built in video conferencing capabilities. The data is encrypted and sent 
to the user on the other end of the video conference and that individual 
can voice their opinions about the issues at hand. 

Physics based computing — Many modem designs are created with 
physics based computing methods or they desire the ability to analyze 
data with this capability. The CoVE and associated hardware will 
enable this high-fidelity computing to be accomplished at ground- 
breaking speeds. 

Integrates decision makers to the design process 


The CoVE Vision 


• Collaborative Design Environment 
with Advanced Visualization 

• High-fidelity Conceptual Design Tool 
at near real-time results 

• Real-time analysis of problems 

• Ability to design collaboratively with 
team locally or across geographically 
dispersed locations 

• Integrated design tools 

• State of the art visualization 

• Multi-disciplinary optimization in a 
collaborative environment 

• Physics based computing 

• Integrates decision makers to the 
design process 




distributed collaboration 


ASDL CoVE- March 2004 
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Finally, this is a photograph of the CoVE in action during a notional 
review of a morphing UCAV. 


The 18 foot screen uses 12 x 64 inch plasma screens in a seamless 
manner to convey information in a high-resolution manner. This is 
useful because the massive amounts of information generated in 
these collaborative designs are difficult to visualize and the advanced 
visualization capabilities of the ASDL are looking for new ways to 
present the information so that all relevant information can be seen at 
once. Shown in this notional example are the following: 

1) The ARIS flowchart and data management tool. ARIS allows the 
storage and retrieval of all program files. These files can be attached 
to the steps of the process, keeping the team organized and allowing 
digital reviews to take place. 

2) A morphological matrix which shows all the possible combinations 
of the design. When a higher-fidelity capability is added to the 
CoVe, it will be possible to truly analyze ALL the combinations in a 
morphological matrix (with first-order analysis) so that trade studies 
can be performed on the fly. Previously, this discrete design choice 
only allowed a single design to be analyzed. 

3) The house of quality tool for the establishment of customer 
importance and multi-parameter interactions. 
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4) A TOPSIS decision making tool with slide-bars that allow the 
decision makers to see the impact of varying the customer 
importance parameters. Before the CoVE, these values were static 
and needed to be generated before the presentation. With the power 
of the CoVE, this information can now be varied on the fly.A 
Powerpoint Presentation. Due to the crisp resolution of the 12 
screens, it is not necessary to display the charts in full-screen mode. 

5) A pareto frontier that represents the locus of optimal points in 
multiple dimensions. Slide bars allow the selection of a concept. 

6) Television and video clips that can be pulled from a repository. In 
this instance, a video showing a IJCAV attacking a convoy of trucks 
is pulled in from the Discovery Wings ™ cable channel. 

7) Teleconferencing. The CoVE is equipped with the latest equipment 
and digital CODECS to allow teleconferencing at any site. The 
CoVE can connect to up to three other sites using its internal 
hardware, or up to 100 other remote locations using the Georgia 
Tech network bridge. This capability can be used to reduce travel 
costs and allow multi-site collaboration in a cost-effective manner. 


For more information, please contact: 

Dimitri Mavris 

Boeing Associate Professor for Advanced Aerospace Systems Analysis 
Georgia Tech School of Aerospace Engineering 
Director, Georgia Tech Aerospace Systems Design Laboratory 
Dimitri.mavris@asdl.gatech.edu 
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Space Transportation System Innovative Design 
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This presentation provides perspectives on innovative design of Space Transportation 
Systems. The authors are retired NASA engineers/managers, who are employed by AI 
Signal Research, Inc. in support of NASA/MSFC Employee and Organizational 
Development Division, providing knowledge transfer to less experienced personnel. A 
primary focus has been the engineering design process, its characterization, teaching, and 
improvement. 
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The presentation addresses the current design process for Space Transportation 
Systems (STS), its characterization and shortcomings, and discusses approaches to 
improve the process. The emphasis is on those innovative improvements that can be 
achieved in the near future. 


AGENDA 

Current Design Process 
Characterization 
Shortcomings 

Innovative Improvements (Achievable in near future) 
Functional Relationships - Next Major Step 
Integrated Performance Model 
Sensitivities and Margins 
Risk Prediction and Probabilistics 
Communication and Information Systems 
Other Areas 

Concluding Remarks 
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To begin with, space transportation systems are very complex systems, with 
many diverse parts and associated interfaces. They have extreme performance 
requirements and high power densities. This means that they are very interactive, 
and are sensitive to small variations. Consequently, design of such systems is 
itself a complex process. It involves dividing (compartmentalizing) the design 
activity into parts, then reintegrating the parts into the complete system. It is in 
the reintegration and the interfaces where most problems have occurred. 

Furthermore, the process is sequential and iterative, involving many steps. 
Much communication among the many entities is required throughout the process. 
Because of the complexity of the design process, the resulting product has 
shortcomings which need to be overcome. 

We have developed a symbolic model of the current STS design process in 
order to improve understanding and to serve as a basis for improvements. It has 
been taught to groups of engineers and managers; is the basis for a prototype 
learning module at the ODU Advanced Engineering Environments Center; and has 
been applied in conceptual design . 


CURRENT DESIGN PROCESS 
FOR SPACE TRANSPORTATION SYSTEMS 

Space Transportation Systems (STS) are very complex 

Complex vehicles, operations systems, payload accommodations 

Current STS design process is complicated, involving 
Compartmentalization 
Reintegration 
Sequential iteration 
Pervasive communications 

Resulting system has consequent shortcomings 

Symbolic model of current design process 

Developed to improve understanding of process 
Has been taught to groups of engineers and managers 
In prototype AEE Learning Module 
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This figure gathers key elements of the design process model onto one chart. 
The left center diagram indicates compartmentalization of the system into 
subsystems, design functions, and discipline functions, and its subsequent 
reintegration. Compartmentalization has its basis in the divisions of industry, 
government, and academia (top left). The “T-model” (lower left) indicates the 
philosophy of integration. In the center, the compartmentalization/reintegration 
process implementation is illustrated by a subsystem tree and a stack of design 
function planes. The design function planes contain the discipline functions. 
They are expanded on the right, along with decision gate diagrams. Vertical 
information flow conduits on the stack connect the design functions. Information 
flow matrices are shown below the stack; the left being an “Ixl” matrix for 
subsystem interfaces, and the right being an “NxN” matrix for information flow 
among design functions and discipline functions. The top diagram represents how 
the design is iterated through trade studies to achieve the best design for the 
integrated system. 


We have called this process “Technical Integration.” which is a significant 
feature of the symbolic model that characterizes the design process as described 


in NAS A-TP-200 1-21 0992 . 


Design Process Characterization - Symbolic Model 


Technical Integration 



Ref. NASA TP-2001-210992 
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Planning, control and documentation of the process are provided by classical 
Systems Engineering (CSE), represented here by the Systems Engineering “V”. 
The “V” indicates the product life cycle from requirements through design and 
manufacture, then verification of the subsystems and the system, and finally, 
systems operation. CSE also enforces project process commonality among 
projects in a program, while providing discipline associated with design products 
and processes during technical integration execution. 


Design Process Characterization - Symbolic Model 

Engineering The System 


Technical Integration 


Understand User Requirements, 
Develop Systems Concept 
and Acceptance Plan 



Develop System Performance 
Specification and System 
Verification Plan 


Expand Performance Specifications 
Into Cl “Design-To" Specifications 
and Cl Verification Plan 


\ 



Demonstrate and 
Validate System to 
User Acceptance Plan 


Integrate System and Perform 
System Verification to 
Performance Specifications 


Assemble Cl's and Perform 
Cl Verification to Cl 
“Design-To” Specifications 



Evolve “Design-To” Specifications 

Inspect to 

Into “Build-To” Documentation 

“Build-To” 

and Inspection Plan 

Documentation 


/ 


Fabrication Assembly and Code to “Build-To” Documentation 


Systems Plane Responsibility: Ensure the system will 
satisfy all requirements and constraints for the entire life cycle. 
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Overlaying the classical Systems Engineering “V” on the Technical Integration 
process then represents the total System Design Process. 


Design Process Characterization - Symbolic Model 


System Design Process 
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The symbolic model has aided understanding of the process, has helped 
participants understand where they fit in the process, and has potentially reduced 
problems in the design through lessons learned. The lessons hopefully help 
avoid some of the problems of the past. Some experienced practitioners have 
affirmed the model’s applicability [usefulness], saying, “Where was this model 
20 or 30 years ago when I needed it?” 

However, there are major inh erent shortcomings in the current process, 
including lack of fidelity in the conceptual design phase, fragmentation of the 
process leading to potential interface problems, difficulty in designing for cost 
and the “-ilities” (reliability, operability, etc.), difficulty in predicting risk, and 
inability to optimize the total system. These shortcomings result in product 
consequences that include performance shortfalls, unforeseen interactions and 
sensitivities, unanticipated failures, and high operating costs. 

Overcoming these problems will require innovative approaches. 


CURRENT STATUS AND FUTURE NEED 


Symbolic model of current design process has been useful in 
+ Aiding understanding and clarifying roles 
+ Reducing problems associated with the design 


However, significant shortcomings remain: 


PROCESS SHORTCOMING 

- Lack of fidelity in concep 



PRODUCT CONSEQUENCES 

- Performance shortfalls 

- Unforeseen interactions 


design stage 


- Difficulty designing for cost 


and sensitivities 

- Unanticipated failures 

- High operating costs 


and -ilities 


- Difficulty predicting risk 


- Inability to optimize the total 


system 


Overcoming these problems will require innovative approaches 
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Here are some illustrations of the lack of fidelity in the conceptual design 
phase — concept vs. reality. The picture on the left shows Shuttle orbiter 
maintenance as envisioned during its conceptual design. The picture on the right 
shows it as it is today. 

In the area of performance the Shuttle was intended to carry 65 Klbs. payload 
to low earth orbit. The first flight could achieve only 20 Klbs. Subsequent 
upgrades have increased the delivery capability, but it has never achieved the 
originally-intended 65K. 


The Saturn V design originally had four engines on its first stage, which 
analysis showed to be sufficient to meet the vehicle’s performance requirements. 
Some far-sighted persons added a fifth engine for margin, which enabled the as- 
built Saturn to perform its missions, including the Skylab mission. 

More recently, the Access to Space project (which was never produced) 
predicted a dry weight of 160 Klbs. at conceptual design. An activity which put 
more detail into the vehicle description resulted in a more realistic dry weight 
prediction that was greater than twice the original value. 


Cost growth is notorious. Two examples are illustrated here. STS costs are a 
strong function of the flight rate. 


CONCEPTUAL-TO-ACTUAL SNAPSHOT 


Performance 


Saturn V 

Space Shuttle 


Access to Space - 25k Payload 

-SIC Initial 4 FI engines 

- Payload Req. 

65K 


-Initial Dry Weight 

160k 

- Payload Req. 85K 

- Flight 1 

20K 


(conceptual designer) 


- Added another FI engine 

-Flight 8 

35K 


-Refined Dry Weight 

354k 

- Payload capability 100K+ 

-Flight 100 

47K 


(detail designer) 


Cost 






Saturn V Total Program Cost 


Space Shuttle Cost Per Flight 


- W. Von Braun 

5B 


- 1972 

29M (1985$) 


- B. Holmes 

10->13B 


- 1985 

273M 


- J. Webb 

20B 


-2000 

373M (8 Flts/yr) 


- Actual 

20->40B 


-2004 

600M 



Operations 




Conceptual Actual 




no 



The system attributes for which we design can be represented by sets of 
interacting metrics. They can be collected in to three groups: 

1. Performance, which is the physical behavior of the vehicle — its payload to 
orbit, its accuracy of delivery, etc. Typical performance categories and metrics 
are shown in the left box. 

2. The “-ilities”, which include reliability, operability, safety, etc. Some 
typical “-ilities” metrics are shown in the center box. 

3. Cost, which includes various cost metrics, typical of which are shown in the 
right box. 


These metrics are interactive both within and among their categories. 
Changing one of them in the design will affect others. 


INTERACTING METRICS OF DESIGN 

(TYPICAL) 


PERFORMANCE 


-ILITIES 


COSTS 


Propulsion 


Reliability 



System Efficiency 

-Isp 
- Thrust 


- Margins 

- Sensitivities 

- Redundancy 


Life Cycle Cost 

-T/W 


Safety 



- # Engines 


- Hazardous Materials 

- Hazardous Operations 


Development Cost 

Dry Mass Efficiency 


Operability 


- Staging 


- Touch Labor 



- # Elements 


- # Parts 



- Geometry 

- Structural Concepts 

- Load Paths 

- Materials 

♦ 

- # Interfaces 

- Assembly / Checkout 

Manufacturability 

- Complexity 

+ 

Infrastructure Cost 

System Efficiency 


- Process Diversity 


Sustaining Operations 

- Protection from 


- Facility Availability 


Environment (TPS) 

- Target Accuracy 

- Avionics / Software 

Efficiency 


Flexibility 

- Mission Changes 

- Block Changes 


Cost Per Flight 


- Technology Upgrade 



Loss Management 
Efficiency 

- Interactions 


Reusability 

- Inspections 

- Refurbishment 


Cost Per Pound 

- Aero Shape 


- Durability Design 



- Drag / Gravity Losses 


-- Fatigue 



- Unknown Unknowns 


-- Fracture 
-- Wear 
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In order to design for these interacting metrics, the current design process 
involves a sequential, iterative approach. It begins with requirements, 
hypothesizes a concept (an architecture), and predicts the attributes of the concept. 

First, the physical performance is predicted. The predicted performance 
probably does not meet requirements, so iterations on the concept design 
parameters are made until reasonable convergence is achieved, or else the concept 
is discarded. Typically, the “-ilities” are predicted after the performance. Again, 
iterations on the concept are made, along with updated performance predictions. 
Likewise sequentially with cost predictions. 

Performance predictions are somewhat uncertain, but “-ilities” predictions are 
much more uncertain. By the time we get to cost prediction, the view is very 
unclear. 

This sequential iterative uncertain process is inefficient and time consuming 
and produces designs that have major shortcomings. The process must be 
improved. 
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What would be an ideal design process? We might imagine something like the 
cartoon, where the requirements go in, and the design specifications come out. 


Even if we wanted this “ideal” process (and there may be reasons to not want 
it), it would be very far in the future. 


We will focus on innovative improvements that are achievable in the 
nearer-term. 


"ideal" procsss 



While this ideal design process is far in the future, we will discuss 
major improvements that can be made in the nearer term. 
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The next major step that should be achievable involves integrating the 
system’s design through functional relationships that connect the attributes of the 
operational system to its integrated systems analysis model. 


Another important feature is the integrated analysis model which combines 
hardware descriptions, design functions, discipline functions, and inherently 
accounts for their interactivity through a combined formulation. The model 
predicts the operational attributes. It connects through the CAD and CAM 
systems to the realized design product. 
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The functional relationship idea is illustrated by a two-way mapping between the 
Concept (architecture) and the Design Solution space. The space is n-dimensional 
but we illustrate it in three dimensions that represent the performance variables, the 
“-ilities” variables, and the cost variables. 

The forward functional relationship arrow represents prediction of the concept 
attributes. The more powerful direction is the backward arrow representing the 
inverse functional relationships, which map the measures of performance, -ilities 
and cost onto the design variables of the concept. 

These functional relationships are not necessarily easy to obtain, but we need to 
work toward having functional relationships that bring measures of performance, 
“-ilities” and cost onto the designer’s table. 


FUNCTIONAL RELATIONSHIP APPROACH 



Performance 


t 


Co. itcpi 



Design Solution 
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Concepts (architectures) that differ significantly map onto different surfaces in 
the solution space. A surface represents the effects of varying the design variables 
(such as skin thickness, engine thrust, etc.) for each concept. The region of 
validity of each concept is represented by its surface. 

The difficulty associated with the design process is further complicated since 
we usually design new launch vehicles to conflicting needs (requirements) among 
the military, private sector, and NASA. This usually leads to multiple 
architectures. Now combining this complexity with the functional relationships 
leads inherently to a map of multiple concepts (architectures). 


FUNCTIONAL RELATIONSHIPS ENABLE DESIGN 
SOLUTIONS FOR MULTIPLE CONCEPTS 



t 


Performance 


Concept B 


Design Solutions 


Concept A 
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Requirements would map into this space; for example, consider various 
design reference missions (military, private sector, or NASA), where each might 
group into a set. We illustrate two sets in the figure, represented by the red and 
blue dots. The first vehicle would capture the red set but not the blue set. It 
would take another vehicle to capture those missions. 


MAPPING OF SOLUTIONS TO REQUIREMENTS 

4 Performance 

▲ 


• •• 






c. Mapping of Conceptual Solutions onto DRM Requirements 
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An integrated performance model is another important component of 
improving STS design. A group at MSFC called the Vehicle Integrated 
Performance Analysis (VIP A) Team has accomplished significant progress in 
increasing design fidelity and reducing turn-around time. 


Based on the symbolic model, it employed a backbone analysis that integrated 
subsystems, design functions and flight sciences such as aerodynamics and 
performance/trajectories. The backbone model provided a connector for 
discipline-specific models. It is based on a parametric CAD model — level of 
modeling fidelity. 


It has been applied to several projects where it brought the level of fidelity 
forward in conceptual design and identified issues not otherwise found. 


Much progress was effected by the team’s relationships and its adapting to the 
integrated environment. 


INTEGRATED PERFORMANCE MODEL 
AND INCREASED FIDELITY IN CONCEPTUAL DESIGN 


Initial Step: Vehicle Integrated Performance Analysis (VIPA) 



• r 

w 


• Increased design 
fidelity 

• Integrated design 
functions, subsystems, 
and flight sciences 

• Shortened trade 
study times 

• Enhanced 
communications 

• Stimulated 
improvements in 
discipline functional 
relationships 


Enabled by symbolic process model, integrated computer modeling, 
and team cultural adaptation 
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VIPA has taken initial steps, but the goal is a fully integrated high fidelity 
performance model that would integrate the multiple areas and avoid the 
problems of hand-off, interfaces, and iterations. This model would depart from the 
those used in the recent past where point forces, mass properties derived from 
mass estimating relationships, and load/thermal indicators were used. The 
advanced model would include structural CAD models, distributed coupled 
aerotherodynamics including plume effects, rigid-body/elastic-body structural 
response, wind profiles, dynamic characterization of GN&C systems, flight 
performance optimization, and so on. 



INTEGRATED PERFORMANCE MODEL 

GPS 


GN&C 


Avionics 


Structures, 


Materials 


Propulsion 


Tra jectory/ Perform a nee 


Thermal Systems 


HEAT 


WIND 


Aerothermodynamics 


Natural Environment 


Beyond the initial step achieved by VIPA, the need is to fully 
integrate the design functions and disciplines into a high fidelity, 
concurrent, interactive performance model 
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Another area generally lacking is determining design sensitivities, uncertainties, and 
providing adequate margins. It is crucial to know the sensitivity to the system being 
designed, and to provide adequate margins in light of uncertainties. 


Innovation in this area can be achieved through the application of functional 
relationships and associated modeling error functions in conjunction with the statistical 
characteristics (means and variances) of all the input and coupling variables. The 
functional relationships plus modeling error functions would provide information relating 
the outputs of subsystems with coupling variables, subsystem specific design variables, 
and subsystem sharing variables. Calculation of subsystems output mean values and 
variances could be accomplished automatically. The mean values of the subsystems output 
variables could be approximately determined algebraically by applying the functional 
relationships plus modeling error functions in conjunction with the mean values of all the 
input and coupling variables; the second partial derivatives of functional relationships and 
modeling functions; and the variances of the input variables, coupling variables, and 
modeling error variables. The variances of subsystems output variables could also be 
determined algebraically in terms of the sensitivities of the functional relationships and 
modeling error functions with respect to all input and coupling variables in conjunction 
with the variances of input variables, coupling variables, and modeling errors. After the 
subsystems means and variances are determined, engineering judgments would be made 
regarding their application (experience based: one sigma, two sigma, or three sigma) in 
conjunction with a safety factor to assess reasonable margins with respect to the statistical 
determined allowable limit (experience based: one sigma, two sigma, or three sigma) of 
the capability of the various subsystems. 


SENSITIVITIES AND MARGINS 


Determining design sensitivities and providing adequate margins 
are critical to successful design 


Innovative design features: 

• Automatic calculation of sensitivities 

by models 

• Expert system to guide margin provisions 

• Historical databases of key parameters 
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An essential adjunct to deterministic analysis and design is probabilistic 
description and risk prediction. Since predictions, natural environments, and 
hardware performance are all inherently uncertain, probabilistic approaches are 
appropriate for modeling and prediction. 


Areas for innovation include the application of the functional relationships and 
modeling error functions in conjunction with the probability density distributions 
of all the input and coupling variables. Automated Monte Carlo or numerical 
methods would be used to obtain subsystems attribute distributions. In addition, 
the probability density distributions of the subsystems capability would be known. 
Then risk prediction would be made quantitatively wherever possible. Advanced 
approaches can assist the designer’s judgment in areas known only qualitatively. 


Introduction of new technologies is a crucial issue in design, and innovations 
are needed to assist in technology risk assessment and its mitigation. 


RISK PREDICTION AND PROBABILISTICS 


Prediction of risks and probabilistic characterization 
are essential adjuncts to deterministic design. 


Innovative design features: 

• Automated Monte Carlo of design parameters 
and variables to obtain attribute distributions 

• Risk prediction for performance, cost, schedule 

Quantitative where possible 
Judgment-assisting for qualitative areas 

• Technology risk assessment / mitigation 
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There are a number of advanced interactive information and communication 
systems being envisioned. 


We proposed a readily achievable system based on the compartmentalization areas 
of the current design process, called the Integrated Information and Communication 
System (I 2 CS). It would connect all process participants, provide co mm on and correct 
design information, insure that interactions are accounted for, and particularly 
important, would provide prompts for needed information. 


COMMUNICATION AND INFORMATION SYSTEMS 



Integrated Information and Communication System 
Connects all process participants in the design 
Archives valid/correct design information 
Ensures that interactions are accounted 
Provides prompts for needed information 
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There are other areas for innovatively improving the design process, some of 
which are noted here. 


These include: 

1 . Collaborative engineering tools, which has received much attention. Also 
included would be multivariate decision-making tools. 

2. Concept generation (synthesis) tools, where in ideal terms it would be 
desirable to apply inverse engineering to convert requirements into design 
concepts. In the near term we could look to innovative approaches to stimulate 
the creativity of individuals in generating concept ideas. 

3. Expert systems to aid the designer based on extracting information from 
historical databases, asking probing questions, performing reality checks, and 
providing lessons learned to avoid the problems of the past. 


OTHER INNOVATION AREAS 
include 

COLLABORATIVE ENGINEERING TOOLS 
Visualization aids 
Integrated models and databases 

Virtual presence with concurrent access to graphics and data 
CAD/CAM with electronic inspections 
Multivariate decision-making tools 

CONCEPT GENERATION (SYNTHESIS) 

Inverse engineering to convert requirements to design concepts 
(Far-term ideal process) 

Innovative approaches to stimulate creativity of individuals to 
generate concept ideas (Nearer-term) 

EXPERT SYSTEMS 

Pertinent data collection and information extraction 
Interactions, probing questions, reality checks 
Lessons learned 
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There have been benefits of applying the symbolic description of the current 
design process; however, there is a need to innovate and significantly improve the 
design process. 


Major improvements in fidelity and efficiency should be enabled through 
computer capability. 


The improvements identified are reasonable to achieve in the near-term. 


However, there are hurdles to be overcome in achieving the next major step in 
design process and tools. These include 

- Defining Functional Relationships 

- Integrating all the elements of the design process 

- Obtaining input information for probabilistic design 

- Cultural change to embrace the next major step 


CONCLUDING REMARKS 

• Symbolic process description has enabled understanding and improvements 

• Computer capability exists to improve fidelity and accelerate design process 

• Near-term improvements are achievable 

• Main hurdles in achieving next major step in design process and tools: 

- Defining Functional Relationships 

- Integrating all the elements of the design process 

- Obtaining input information for probabilistic design 

- Cultural change to embrace the next major step 
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Form Follows Function and Physics: 

Simulation Based Optimization Drives the Shape of Tomorrow’s 

Aerospace Products 


Alex Van der Velden 
Engineous Software 
Atlanta, GA 
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In this talk we will discuss the application of commercial process integration and 
design optimization tools in aircraft design from small bizjets to large commercial 
airliners. 


These tools allow us to generate aircraft designs directly based on economic and 
performance objectives as computed by high fidelity physics models. 

We will also take a look at the future, were integrated processes will be deployed 
across geographic and enterprise borders using the FIPER software. 


This talk 


♦ Desktop PIDO (iSIGHTj has been used worldwide in 
product development for over 10 years: 

♦ Desktop Tools 

♦ Large turbofan 

♦ Small Bizjet 

♦ Regional Jet 

♦ B2B PIDO (FIPER) is the shape of things to come. 

♦ Airbus VIVACE 

♦ GE & Parker Hannifin. 


♦Engineous ^ 

^ ^ softwares 


Engineous Software Proprietary. Not to be reused or distributed without the approval of Engineous. 
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Engineous Software has two commercial PIDO (Process Integration and Design 
Optimization) products: 

iSIGHT executes simulation-based design processes, including commercial CAD/CAE 
software, internally developed programs, and Excel spreadsheets on the engineer’s 
desktop & local network. It provides leading edge design exploration and optimization 
technology to ensure that an optimal design is discovered that meets or exceeds all 
customer requirements. 


FIPER stands for “Federated Intelligent Product Environment”. This allows the 
ability to share models between organizations, so partners can execute each others 
design processes in a geographically dispersed, secure mode, without exposing 
proprietary company data. 


Engineous PIDO Product Lines 


| — |” |~ Development Application 

oo Automates the iterative design process 
oo Integrates the numerous codes involved in the design 
process so that they can be utilized in a single run 
oo Optimizes the design for user-defined parameters (e.g. cost, weight) 

F4PER Development Infrastructure 

oo Integrates disparate 3 rd party or home grown 
applications and provides a common GUI for them 
oo Handles data exchange among the applications 
oo Provides workflow across applications 
oo Provides a framework for knowledge-based engineering 
oo Provides a true collaborative engineering design environment 
inter- or intra-enterprise 

oo Retains and protects a company’s intellectual property through federated 
process execution 




♦Engineous 

~ ^software; 


Engineous Software Proprietary. Not to be reused or distributed without the approval of Engineous. 
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The PIDO software category was started by three company’s in the South East 
United States in 1995. My own company Synaps, Inc. was started in Atlanta and 
originated out of the Airbus in Europe. Its product were Pointer and Epogy. Synaps was 
sold to Engineous software in January 2004 and currently Epogy and the Isight product 
are merged into one package. The examples that I will show in this presentation will be 
a mix or projects done by Synaps and Engineous Software. Engineous was founded in 
1995 by Siu Tong and originated out of software developed at General Electric. 


Phoenix Integration’s Model Center software was founded by Brett Malone in 
Blacksburg VA and is mainly used at NASA and other government agencies. 


Engineous Software is currently the market leader with over 50% market share, but 
currently there are a dozen software companies competing for this emerging market. 


Desktop PIDO tools 


4 PIDO: Process integration and design optimization 

4 Was started in the SE USA 

♦ 1994 Pointer/Epogy Synaps, Atlanta (now part of 
Engineous) 

♦ 1995 Isight Engineous Software, Cary NC origin GE 

♦ 1995 Model Center. Phoenix Integration, Blacksburg VA 
origin NASA 

4 And are used world wide for the development of 
everything from P&G diapers to GM cars, but has its 
roots in aerospace. 
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One of the first applications of Engineous Software (before it was 
commercialized) was on a large turbofan engine in 1991. Within two weeks the 
optimization of the turbine disks allowed one stage out of seven to be removed. 
Without the software this would not have been possible. This saved a quarter a 
million dollars per engine, reduced the sfc and made the engine 250 lbs lighter. 


Large Turbo Fan Engine 


r 

♦ Investment: 2 months to develop appl., 2 weeks to run case 

♦ ROI: 

• Savings of $250,000 per engine 

• $500M total savings over 2000 units 

• 1% lower SFC, 200-250 lbs lighter 


Scientific America 
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One the other side of the Atlantic Airbus used our software to support its 
very large transport aircraft development. (Aviation Week Feb 22, 1999) The 
software was developed & used over a 5 year period and created significant 
insight in how to reduce the weight of such a massive aircraft. 


MDO Large Transport Aircraft 



Engineous Software Proprietary. Not to be reused or distributed without the approval of Engineous. 


♦Engineous 

^ softwa re ; 
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In this case the physics was modeled as carefully as possible using computational fluid 
dynamics and structural codes. The problem was to find the right exterior and interior shape in 
order for the wing structure + fuel to be minimal for the design range and payload. 


The design team faced a lot of uncertainties. For instance we did not know exactly what level 
of bias/error to expect from the computational codes. These expected biases were captured in 81 
scenarios. Each scenario was considered equally likely. To goal was to find a solution that was 
good whatever the right scenario was. 


7 top level design variables were selected to represent the wing planform, thickness and 
spanwise lift distribution. The spanwise lift distribution determined the structural optimization 
and we were therefore more or less able to decouple the structural and the aerodynamic 
optimization of the problem. 


MDO Multi-level Physics 


■ Minimize aircraft to mass using (Epogy) + combined 
proprietary simulations 

■ By varying 7 global parameters for 81 scenarios 

■ span wise lift distribution 

■ wing planform (aspect ratio, chord distribution, sweep, taper) 

■ wing thickness distribution 

■ By iteratively holding the global design constant and 
varying 100 detail parameters 
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For each of the 81 scenarios the the multi-level optimization loop including 
detailed structures and aerodynamics was completed. 

It was interesting that all of the solutions showed basically the same aspect 
ratio wing and that whatever we assumed it should be possible to save at least 
10,000 kg. 



16000 



0 2000 4000 6000 8000 10000 12000 14000 

Savings in Structural Mass (kg) 


>At least 10 tons can be saved independent on scenario 


♦Enqineous 

” ^softwares 



A 
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We were interested how the optimum of the top design variables was 
affected by the different scenarios and found that the aspect ratio of 8 was 
definitely the right answer. However, we also found that it was probably better 
to increase the sweep a few degrees and get a totally different non-elliptical 
wing loading. 


MDO Identify Design Drivers 


80/20 rule in action 
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changes have to be made simultaneously 


♦Engineous 

” O software* 


Engineous Software Proprietary. Not to be reused or distributed without the approval of Engineous. 
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This result was non-intuitive and cannot be obtained by sequential ID scans 
of the design space. 

For instance, starting from the baseline wing with aspect ratio 8 reducing the 
aspect ratio to 6.5 actually improves the max takeoff weight of the aircraft. The 
reason is that the baseline wing was rather thin and the structural benefits 
outweigh the increase in drag. If from that point I would do a scan varying 
thickness I would simply confirm that the current thickness was optimal. . . 

If I vary the aspect ratio of the optimized wing I can indeed show that it is in 
a local minimum. Showing local minima is a good application for ID scans. 



r 


A3XX-old 


Optimized Planform 


Total Mass versus AR 


Total Mass versus AR 



14000 ' 


5.00 6.00 7.00 8.00 

AR 


9.00 


5.00 


6.00 


7.00 8.00 9.00 

AR 


> shows 1-D scans CANNOT be used to optimize, but 
only to show optimality 


♦Enqineous 

^ U software- 
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Years earlier Airbus had considered this as a possibility. The benefits of unloading the tips 
were numerous. 


Less root bending moment created less wing structural weight and an nose up pitching 
moment that reduced the need for more tail down load (typical for statically stable aircraft). Less 
tail down of course requires less lift on the main wing for the same upwards force and thus we 
save not only drag, but also wing weight. 

The real problem was the fact that in previous studies the increased inboard loads caused the 
transonic drag to rise to unacceptable levels. 


Impact of Wing Load Distribution 


r 



wing span 

Reduce Weight 

^ Less root bending moment Hess weight 
>-Less tail down ~ less load Hess weight 
^Creates inboard transonic drag problem 


♦Enqineous 

^ ^softwares 


Engineous Software Proprietary. Not to be reused or distributed without the approval of Engineous 
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Not so here. This study showed that it was possible to increase the inboard 
loads without increasing the transonic drag. 

The detailed aerodynamic shape optimization had shifted the location of 
maximum thickness-to-chord further aft. In effect the maximum thickness iso- 
line was now more swept and the wave drag was reduced. Wing optimization 
put area-ruling in the wing whereas on the B747 it is done by putting the hump 
on the fuselage. 



r 


o 



I "^maximum 
^ thickness 


span 


Reduce Drag 

^New ‘area-ruled’ inboard design 
>-More sweep 


Hess drag 
Hess drag 


♦Enqineous 

^ O software i 
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In recent years most aircraft builders have started to use PIDO software. As 
computer power is going up and CFD calculations are faster it is now possible 
to shape optimize CAD geometries directly with coupled commercial CAE 
tools. 


MDO: Small Business Jet 



Wing & Underbody "Stretched" Fairing Base Model 

Translate Vertically 



Full Configuration 


♦Enqineous 

~ 'J software) 


Engineous Software Proprietary. Not to be reused or distributed without the approval of Engineous. 
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In one such application a well know business jet manufacturer was interested 
in the ‘best’ wing fuselage intersection design. For a given amount of aisle 
height we can attach a thick low weight wing center section with a large fairing 
or a thin wing center section with a small fairing. The fairing has both weight 
and drag. The picture above shows two fairings with extreme values of the 
fairing height. 
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In this application the aerodynamic and structural problem were decoupled 
using the fairing height. For a given fairing height an optimal fairing was 
designed which produced a computed drag and which had a known weight. 



♦ Enqineou 

’-'softwares ^ 
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For a given fairing exterior height we still had the option of having a low 
center section beam and a circular fuselage center section and a tall center 
section beam and a non-circular center section. In one case the fuselage was 
light and the center section wing was heavy and in the other case it was the other 
way around. 


Structural Optimization 



Initial design 
Circular cross-section 
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The structural optimization had two steps. The optimum fuselage non- 
circularity and beam height were computed for an optimal structure. The 
optimal structure was calculated by optimizing the thickness of the structural 
elements for minimum weight for dozens of fatigue load cases. 


Structural Optimization 




Local variable optimization 
Circular cross-section 
Stress constraints satisfied 
Mass = XXX lbs 
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In the end the optimum overall shape was somewhere in the middle and it 
was some 3% lighter than the baseline configuration. 


Structural Optimization 


r 



Global variable optimization 
Non-Circular cross-section 
Stress constraints satisfied 
Mass = -3% 
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The Bombardier company of Canada has pushed the envelope even further 
with our help. They directly optimize internal structures, aeroelastics, wing 
profile and planform shape for optimal aircraft performance and economy. 
(Daratech Aero 2003) 


MDO: Regional Aircraft 



■ The objective of coupling aerodynamic and structural models is to 
compute the flow over flexible wings 




BOMBARDIER 

AEROSPACE 


MHUUDlin 
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This shows that it was possible to start with a rectangular wing of aspect ratio 
2 and optimize it to a very high performance wing with sweep. It must be noted 
that all of this required a new NEC supercomputer and thousands of iterations. 

This study showed that higher wing sweeps than employed conventionally 
may be better for regional aircraft. 


Automatic Design of Structure & Exterior Shape 



Initial configuration - 

AR =2 
L/D -9.5 

A M_fuel - +2100 kg 

AM_structure- 0 kg 
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Optimized configuration 

AR = 8.9 
L/D - 20 

A M_fuel - -2160 kg 

AM_structure- +290 kg 
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Today the question is no longer “will it work for our application”, but more 
“will it work for our enterprise.” 

The scalability of PIDO software solutions is at the core of Airbus’ VIVACE 
program. 


... the next level 


Value Improvement through a Virtual 
Aeronautical Collaborative Enterprise 


Total funding €72M 
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The VIVACE program ties together Airbus with its suppliers. One of them is GE. 
The same engine core that is used on the GE-90 is planned to fly on the A3 80. 


IBM and Engineous were selected as infrastructures partners for this program. The 
reason for Airbus to select Engineous is its new FIPER software. 



A380 - Super-jumbo Jet 


+55 Airbus suppliers 

♦ IBM and Engineous FIPER selected as the 
infrastructures for the program 

♦ Implementation has started 
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FIPER allows you to deploy integrated processes across geographic and enter- 
prise borders. This allows partners to share models securely without exposing each 
others proprietary company data. 



♦ Problem solving collaboration among business partners and 
technology providers 

♦ Agile product development to react to product trends 
and market needs 

•Enqineous 
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FIPER is process integration at an enterprise level and intra-enterprise level. The 
component based java architecture reduces the number of “knowledge gaps” which 
allow the components to be reusable by people other than the original author. This 
software also requires middleware (such as web sphere) and is not intended for the 
engineers’ desktop. Engineous has teamed up with IBM in order to implement FIPER 
solutions. 


Fiper Solution 


♦ Fiper is a brand-new commercial product (version 
1.5), not a research technology. 

♦ Fiper is an integration framework for collaborative 
product development 

♦ Fiper has a component-based architecture for fewer 
“knowledge gaps” 

♦ Fiper can be deployed across enterprise and its 
suppliers 

♦ Fiper is implemented by IBM 


^Engineous 
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Because FIPER touches many aspects of the design process, several types of users 
need to interact with the system using various clients. Users include systems 
administrators using the system monitor , component developers using the component 
generator and the model assembler using the design gateway. The end user will 
interface through a web top or the design gateway. 

The FIPER ACS is the heart of the FIPER infrastructure. It is the controller that 
manages the internal operations of the system. 


Finally FIPER stations are the distributed network of computing resources that are 
aware of the structure and operation of the FIPER architecture. 


How It Works 


FIPER— A gateway to resources throughout the 
engineering enterprise 


FIPER ACS: The heart of the FIPER infrastructure. 
The controller that manages the internal operation 
of the FIPER system 


FIPER 

INTERFACES: 

Because FIPER 
touches many aspects 
of the design 
process, several 
types of users will 
need to interact 
with the system. 
Engineous provides 
various clients, 
or interfaces, 
to support the 
needs of these 
different users. 



|1 Component 
VV Developer 


FIPER STATIONS: 
The distributed 
network of 
computing 
resources that 
are aware of the 
structure and 
operation of the 
FIPER architecture 


FIPER COMPONENTS: The tools, methods 
and process models that provide application 
services in the FIPER environment 
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FIPER allows you to integrated and make interoperable the tools that are 
utilized in the design and analysis environment. It allows you to collaborate with 
geographically dispersed team members, business partners and supply chain 
providers. It allows you to perform design and analysis across a global network 
regardless of platform, software, company or country. 


FIPER Allows You to... 


♦ Integrate and make interoperable, key tools utilized in 
the design and analysis environment 

♦ Collaborate in real time with geographically dispersed 
design teams, business partners, & supply chain providers 

♦ Perform design and analysis across a global network, 
regardless of platform, software, company or country 

♦ Lower hardware investments through effective use of 
legacy systems and more efficient job distribution 

♦ Eliminate mundane, iterative tasks by automatically 
managing the execution of applications 

♦ Manage design processes and reuse knowledge 
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FIPER takes a standards based no-proprietary approach. It was developed by 
government backing from the National Institute of Standards during a 
$25,000,000 development program. Major manufacturers such as GE and 
Honeywell participated in the development. 

FIPER is a next generation tool for the next generation of aerospace 
products. 


FIPER Key Differentiators 


4 Unlike virtually all competition, FIPER takes a standards based, 
non-proprietary approach 

4 Government backing - NIST ($25 M development) 

4 Major manufacturers are participants in the development 
• GE, Parker, Goodrich, Honeywell, Ford, Rolls Royce, GM, etc. 

4 No competitors offer process integration, collaboration, and the 
Six Sigma engineering tools provided by FIPER 

4 Engineous expertise in this field - market leadership 

4 Engineous has a “head start” on the competition 
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About a dozen pilot programs are already completed with the FIPER 
architecture. In this example GE has modeled the combustor of a jet engine with 
Unigraphics CAD software. Parker models its nozzle using the ProE CAD 
software. 


When GE modifies its combustor, the FIPER system automatically asks 
Parker for a modified nozzle. Based on the changed requirements, Parker 
generates a new nozzle design using a knowledge based engineering tool. A 
modal analysis of the nozzle is then performed to validate the design. 


Application Case Studies - GE / 
Parker 


r 

Nozzle Combustor Pilot 



♦ GE Combustor - UG 

♦ Parker Nozzle - Pro/E 

♦ GE updates combustor 

♦ GE notifies Parker, 
sends geometry 
parameters - asks for a 
new nozzle 

♦ KBE tool creates a new 
candidate design 

♦ Candidate validated 
with modal analysis by 
Parker 
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Summary 


♦Process Improvement is the “Final Frontier” in cost 
reduction, profit improvement 

♦Many companies have taken initial steps in this 
direction, but have not gone the distance 

♦iSIGHT, with more than 200 companies using it, has 
a proven track record in this area 


♦FIPER is far ahead of the competition in addressing 
this urgent business requirement 
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Today High fidelity analysis integrated with PIDO tools, such as iSIGHT 
allows the assessment of radically new aircraft shapes. 


FIPER makes it possible for aircraft manufactures to share the aircraft 
design processes (and its associated risk) with many partners. 


These developments of reduction of risk through better analysis and 
spreading of risk through partnerships may allow us to develop a next- 
generation of higher-performing non-conventional aircraft such as the oblique 
flying wing. 


With form following function and physics this 
maybe the shape of aircraft to come... 
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Formalizing Conceptual Design 


Because of its influence on all downstream processes, conceptual design is a critical 
part of the design process. The informal nature of information in conceptual design 
provides distinct challenges for formalization. This presentation introduces a view of 
conceptual design based in both descriptive research (what designers actually do) and 
prescriptive research (what, in a rational world, designers should do). This leads to a 
framework for design built around recognizing uncertainty in the requirements for the 
design and selectively reducing this uncertainty as design alternatives are initiated, 
developed and selected. 


The techniques presented include methods for modeling the design/requirement 
‘space’; developments of normative decision theory both for formalizing the selection 
of design alternatives and the refinement of design requirements to support this 
selection; and the adaptation of methods form communication theory to provide rich 
measures of the ‘size’ of a design space. Together, these techniques provide a 
framework for formalizing conceptual design that embraces the uncertainty and 
informality inherent to the process. 
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Why Conceptual Design? 


Conceptual design takes place in the earliest stages of the design process, but 
commitments made here extend far down the product development path. At the same 
time, conceptual design is not a particularly costly exercise. For example, changing 
concepts to improve manufacturability may save more in the production 
design/implementation stage than the entire cost of conceptual design. Anecdotally, 
conceptual design is largely responsible for the success or failure of a project. 

But although conceptual design has the greatest leverage in the design process, its 
informality tends to thwart our efforts at using this leverage. Most conceptual design 
methodologies are ad hoc, developed based on tradition or experience. The question for 
design researchers is whether the best of the ad hoc methods should be formalized, or 
whether the lot of them should be replaced by more normative methods. 


Why Conceptual Design? 


Leverage 



TIME (NONLINEAR) 


Challenges: 

Informality -> ad hoc methodologies -> Formalize or 


Replace? 
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Conceptual Design: Idealization 


From a global process viewpoint, design is the process of finding an arrangement of 
components to satisfy a set of requirements. In evaluating theories for design, 
Yoshikawa suggests an ideal design process: all ‘design’ is done on the requirements 
side; a design ‘oracle’ automatically maps these requirements into possible designs 
implied by a given set of requirements. ‘Loose’ requirements imply many design 
solutions. Designers might choose one of these, or they might restrict the requirements 
further toward generating better design performance. As the requirements are refined, 
the design oracle automatically shrinks the size of the design space. This continues until 
the requirements imply a single design solution - further requirement refinement would 
produce no solution. 


Of note in this process is that the design space implied by a requirement is 
heterogeneous - multiple types of design can often meet the same requirements. As the 
requirements are refined, the design space becomes less heterogeneous - only one type 
of design will end up being chosen. If refinement does not produce a single design, the 
designer has ‘left some money on the table’ in terms of design optimality. 


Conceptual Design: Idealization 


Ideal Design (General Design Theory, Yoshikawa, 1987) 



Design Space: Heterogeneous 

Requirement Refinement: Reduces Design Space 
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Conceptual Design: Process 


The process of conceptual design begins with the customer. Methods such as QFD 
and the House of Quality are typically used to translate customer needs into engineering 
requirements. Requirements themselves come in two main forms: functional 
requirements that specify what a design must do and performance requirements that 
measure how well it accomplishes this function. Functional requirements provide the 
impetus for concept generation; formal ‘German-school’ function-based design 
proceeds from a formal decomposition of the system. Evaluation of the generated 
designs is done according to performance specifications. The results of both processes 
are fed back into the requirement process: if no clear decision can be made among 
alternatives there are two basic paths - reduce uncertainty in the requirements or reduce 
ambiguity of the designs. The former requires refining performance requirements, the 
latter functional requirements. 

Of note in this process are a couple of normative methods for design. Set-Based 
design describes the Toyota design process in which requirement refinement is 
purposely withheld from the designers, forcing them to maintain a greater number of 
possible designs. Decision-based design has focused on the application of normative 
methods from decision theory, primarily in the performance requirement/design 
selection loop. Decision-Based Conceptual Design merges these two methods, 
promoting initial requirement ambiguity while providing decision-based means for 
determining paths toward refining requirements in the context of design selection. 


Conceptual Design: Process 



Customer 
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The Road to Formal Conceptual Design 


Formalization of the process of conceptual design can operate on either of the two 
paths stemming from requirements. On the function side, some formalization has 
already been suggested in the systematic design process of Pahl and Beitz. This 
provides a framework for treating function as the transformation of material, energy, 
and/or information. Others have developed basis languages for use in this framework 
toward making it more computable. Some have extended this work further toward 
automated design generation based on case experience derived from reverse engineering 
existing products. The main issues in this final step revolve around the abstraction level 
at which function is treated as well as the completeness and soundness of the 
synthesized designs. 


From performance requirements, QFD provides a basic input to the process in the 
form of design metrics, directions for improvement for each metric, and a set of targets 
for the collection of attributes. Set-based design operates over uncertain targets, flowing 
them down into subsystem design with increased uncertainty to ensure full design space 
exploration. Rather than using uncertain targets, decision-based design generates a 
value function that can be used to evaluate uncertain design outcomes. Decision-based 
conceptual design (DBCD) works in both modes, evaluating the effect of refining 
requirements in the context of deciding among design alternatives at various levels of 
abstraction. 


The Road to Formal Conceptual Design 


Function-Centered 

Function-Based Design 

(Pahl & Beitz) 

Material = 


Energy — 
Information 


Function 


^Material 

—^.Energy 


Information 

Solution-neutral 
Functional Basis 

Controlled vocabulary 
Neutrality-, Computation+ 
Computational Synthesis 
Building blocks 
Composition 
Issues 

Abstraction 

Completeness / Soundness 


Performance-Centered 

QFD/HoQ 

Establish targets 
Set direction of improvement 
Set-Based Design 

Avoid ‘anchor & adjust’ 
Preserve design space 
Flow targets down 
Decision-Based Design 
Normative selection 
Uncertainty management 

DBCD (Decision-Based Conceptual Design) 

Requirement refinement 
Design space navigation 
Design space discourse 
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Complex Systems 


Coupling is what makes systems complex: requirements are coupled to each other 
within the design space and subsystems are coupled to each other through functional 
interactions. Descriptive of Toyota’s design processes, set-based design provides a 
means for managing complex design processes by propagating uncertain requirements 
into the subsystem level and using the design space generated by subsystem designers to 
get a realistic picture of complex interactions. The options generated at the subsystem 
level are then evaluated by a lead engineer who decides where and by how much the 
uncertainty in the requirements is reduced. 


Requirements 

Overall requirements passed down to subsystem level 

Function 

Subsystems interact with each other 
Set-Based Design (Toyota - descriptive/Ward et al. - prescriptive) 


Complex Systems 

Coupling: 



(^Lead EgrT^) | ^ 
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Set-Based Design 


In set-based design, subsystem designers are required to develop design solutions 
that cover a range of potential requirements. Uncertainty about performance targets 
prompts designers to study the many possible tradeoffs among competing design 
objectives. Set based design is controlled by a powerful lead engineer who moves the 
process forward by selectively reducing requirement uncertainty. Where uncertainty 
greatly influences cost (or lead time or quality, etc.), it is reduced; decisions within 
subsystems whose performance is less sensitive to requirement uncertainty are delayed. 

Ward et al. have developed prescriptive methods for set-based design, focusing on 
the propagation of uncertainty from requirement to design space. While these 
methodologies provide a theoretical basis for set-based design, their implementation 
through interval calculus methods fails to capture the richness of uncertainty 
propagation. Too often, the design space implodes based on small changes in 
requirements. We propose an implementation based on probabilistic modeling to 
propagate uncertainty more accurately from requirement to design space. 


Functional Requirements 

Control 

Successive refinement of targets 

Include ‘stretch’ as part of initial uncertainty 
Refinement = Selection 
Manage design space size 

Methodology 

Combine design/requirement modeling space 
Formalize selection/refinement process 
Measure ‘size’ of design space 


Set-Based Design 


Subsystem Input 

Uncertain Targets 


Subsystem Output 

Set of designs 
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Design Space Modeling 


Building on Yoshikawa’s ideal design model and the methodology of set-based 
design, representing the design space is at the heart of decision-based conceptual 
design. In typical design situations, requirements are evaluated using function of design 
attributes. But as demonstrated earlier, the design space contains distinct concepts, 
heterogeneous both in their detailed description and in the function used to generate 
design evaluations. The intuitively obvious result is that we cannot merely invert the 
design-> requirement mapping to generate new designs because the inverse mappings 
are not functions. 


In addition, the functions that map from design to requirement may be derived from 
physical first principles, in which case the functions are generally well-defined, or those 
functions might be derived from experience, in which case the function represent 
approximations of mappings. Design space models must respond to both modeling 
sources. 


To combine analytical and experiential, and to provide inverse mappings from 
requirement to design attribute, DBCD uses a single joint probability density function to 
model both requirement and design attribute spaces. 


Design Space Modeling 

Combine Requirement / Attribute Spaces 
Requirements are functions of attributes: req(attr) 
Design concepts are heterogeneous: multiple mappings 
=>Design attributes are not functions of requirements<= 
Basis of Requirement Functions 
Experience / Empirical 
Physics 
Proposal 

Joint pdf: P(req,attr) 
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Example: Motors 


The equation below assembles a joint pdf from multidimensional samples of the 
design space. A Normal density function is drawn around each design space sample; the 
sum of these individual pdfs generates the overall probability density function. The 
figure below shows the joint pdf of motor weight and output torque for a set of dc 
electric motors drawn from a catalog. Compared to the single regression line typical of 
other modeling methods, DBCD operates over the joint pdf, capturing the underlying 
uncertainty of the heterogeneous data points. The contours show that most mostors form 
the catalog are relatively low mass/low torque motors. They also indicate a wide spread 
in the possible underlying relationship between the two. Dashed lines show the expected 
value of weight conditioned on torque. Both show a trend consistent with the regression 
in areas where there is a lot of data, although no predefined functional relationship is 
required to bias the result. Where data is more sparse, the method based on a less- 
smooth generalization of the motor data shows greater variation in the model. 


Example: Motors 


P(x) = 
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Engineering Model for Weight as a Function of Torque 
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Example: Force Feedback Mouse 


This plot shows a set of contours for a more manufacturing oriented case of a force 
feedback mouse. Here, the competing objectives are the power transmit through the 
mouse and its spatial volume. It appears that, for low-power designs there is little 
relationship between the two, but at higher power the mouse designs tend to get larger. 


Example: Force Feedback Mouse 


Joint PDF for Slider Crank 



UMBC 

An Honors University in Maryland 


168 


Design Decision-Making 


At its heart, DBCD is a decision process. But a decision process needs options 
among which to decide. Here, we draw on the nature of the design-requirement space to 
identify possible sources of design options and types of decisions. For the discrete 
options required by formal decision theory, discrete design variables present the most 
obvious source of alternatives. Constraining a discrete variable to one (or a subset) of its 
possible values makes a ‘hard’ partition in the design space, deterministically removing 
some options from further consideration. These variables are called classification 
variables due to their general use to define design classes. 


Uncertainty of continuous (or discrete-ordinal) variables could influence the decision 
made among the discrete design alternatives. Refining uncertainty in such performance 
variables might be of value if it makes decisions among classification variables more 
clear-cut. Reducing uncertainty in these variables produces a soft partition due to the 
generalization used to flesh out the design space. 


Design Decision-Making 


How to Refine 
Requirements? 

Classification 
Discrete Variables 
Hard Partitions 
Decisions 
Performance 

Continuous Variables 
Discrete-Ordinal 
Soft Partitions 
Refinement 




45 Watt DC PM Motors 
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Making Hard Partitions 


For determining which values of a classification variable would likely produce the 
best design, we can apply expected value decision making. For this process, we need a a 
function that establishes design value (e.g., an optimization objective, a value function, 
a utility function, etc.), a set of discrete alternatives (i.e. the classification variables and 
their possible values) and a definition of the uncertainty in the problem (e.g., the design 
space joint pdf). For each possible decision, the expected value of the value function is 
calculated by integrating over all sources of uncertainty. The option that maximizes the 
expected value of the design is generally selected at the best option. 


This process is normative: it provides a rational model for selection under 
uncertainty that is insensitive to how design options are packaged, the order of option 
presentation, etc. It provides a rational choice in situations where uncertainty cannot be 
reduced. But expected value decision-making does not reveal whether one alternative 
dominates all others or the situations in which the chosen alternative might be sub- 
optimal. 


Making Hard Partitions 

Expected Value Decision Making 

option* = ma ■x. i \E\y (design \ option ,.)]= ^(design | option^, c}lU j 

Inputs 

Value Function (possibly uncertain) 

Discrete Options (possibly hierarchical): Classification Variables 
Uncertainty 
Output 

Rational Choice 

Issue 

Dominance 


UMBC 

An Honors University in Maryland 


170 



Example: Motors 


In the plots below, models for the mass of 45 Watt motors are plotted for different 
values of three classification variables: frame vs. frameless, ferrite vs. rare earth, brush 
vs. brushless. For each curve, the vertical line denotes the expected value of the pdf. To 
minimize motor weight, EVMD recommends frameless over frame, rare earth over 
ferrite, and brushless over brush. But it is clear than in none of these cases is one choice 
deterministically better than the other. By going with EVDM at this point, the designer 
is taking a chance that the resulting design will be suboptimal. 


Example: Motors 


Probability of Motor Weight for Various Classifications 



Motor Weight (oz) 
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Example: Force-Feedback Mouse 


Another example shows the number of parts (top two plots) and the size (bottom two 
plots) for a variety of force feedback mouse designs. The decisions to be made include 
both the type of position sensor - 2-D optical vs. 1-D encoder - and the type of actuator 
mechanism - slider crank vs. rack and pinion. Again, there is no dominance in the 
depicted decisions: optical and rack& pinion to minimize parts, optical and slider-crank 
to minimize size. 


Example: Force-Feedback Mouse 
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Making Soft Partitions 


To help clarify decisions among classification variables, we can evaluate how the 
resolution of design uncertainty might change the current best decision. If, at all points 
in the uncertain design space decision A is better than decision B, then there is no need 
to further resolve uncertainty. But, if there are ranges of uncertain variables for which 
the best decision is not that indicated by EVDM, there may be value to reducing the 
uncertainty in those variables. The expected value of perfect information (EVPI) is an 
upper bound on the value (expressed in terms of the values function) of completely 
reducing the uncertainty of a variable (i.e., selecting its value at random according to its 
probability density). Where EVPI is high, the designer might seek to resolve 
uncertainty. 


Making Soft Partitions 

Clarify Hard Partitions 

Information Value: Information (a reduction in 
uncertainty) has value if it changes the optimal 
decision 


Expected Value of Perfect Information 

• Narrowing the range of a performance variable is a potential source 
of information 

• Information has value if it might lead to different decisions 

• EVPI is an upper bound on the value of reducing uncertainty: 
assumes it will be reduced to a deterministic value 

• Calculating EVPI does not require design commitment 
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Requirement Refinement 


In the below plot, the expected value of the number of parts given power is plotted 
for two values of the mechanism classification variable: rack & pinion and slider-crank. 
Recall that the rack & pinon is the preferred design from the standpoint of expected 
value; this plot show that, for high system power, the slider crank design is expected to 
require fewer parts. Until now, the designer has made no commitment with respect to 
system power; telling the system the exact power level will help make a better decision. 
Thus, information about power has value and should be pursued by the designer. 


Requirement Refinement 



Knowing how much power the mouse must transmit changes the 
type of mechanism that would be selected based on #parts 

Uncertainty can be explicit (provide a pdf for a requirement) or 
implicit (get requirement pdf from joint pdf) 
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Measuring Design Freedom 


Thus far, we have established methods for determining the value that can be derived 
from placing constraints on the design space in the form of decisions and requirement 
refinement. Generating this design value comes at a cost of design freedom - options 
removed from consideration could have been useful for resolving unforeseen issues 
downstream in the design process. Set-based design is more successful than ‘anchor and 
adjust’ design strategies because delaying commitment affords the design a better 
understanding of the full implications of all design decisions. Preserving design 
freedom is an important part of set-based design, measuring it is the first step. 

A second implication of design freedom has to do with the unintended consequences 
of design decisions. In cases where the value function is complete, it captures all that is 
necessary in evaluating designs. But when the value function is incomplete, designers 
might ‘decide away’ performance for variables not currently in the design evaluation. 
Losing design freedom in these variables means that the designer has made an implicit 
rather than explicit decision. 

Calculating design freedom in a probabilistic framework is a simple extension of 
Shannon’s entropy definition from communication theory. Here, design freedom is 
derived from samples of the design space and normalized with respect to a uniform 
probability density. A design freedom of 1 means complete freedom, design freedom of 
0 implies a deterministic choice. 


Measuring Design Freedom 


Communication Theory (Shannon)! 


Entropy - general measure of disorder in a probability density 
function: 


J Q„ . 


Design Freedom - sampled entropy, scaled w.r.t. complete 
freedom (uniform density): 
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Setting/Refining Targets 


For the motor example, one might want to set a target on motor mass early in the 
project, possibly to reduce uncertainty for other subsystem designers. Absent any value 
proposition for motor mass, one can determine how design freedom drops off the lower 
the mass target is set. For the discrete variables (plot on the left), the dropoff is not too 
severe when constraining the mass to less than 0.20 normalized mass. But for 
performance variables like torque (right plot), design freedom drops very quickly below 
about 0.15 normalized mass. Design freedom can aid in setting constraints by 
illustrating how achievable those constraints are and the degree to which they limit 
future choices. 


Setting / Refining Targets 



Discrete Continuous 
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Setting / Refining Targets 


In this example, a sampled design space derived from 3125 finite element runs is 
depicted. The main objectives are to maximize both torsional and bending stiffness of a 
uni-body structure for a car. The first plot shows that, near the optimum value in the 
upper right there is little design freedom; backing off of the design targets can increase 
the number of ways in which the team can realize them. Other stakeholders, like 
manufacturing, might be interested in the constraints placed on wall thicknesses for 
various values of the design targets; styling might be concerned about the placement of 
the ‘B’ pillar. The below plots might help in the negotiation process as the team sets 
initial stiffness targets. 


Setting / Refining Targets 



Objectives: Max Stiffness 



B-pillar thickness 




umbc B-pillar position 
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Design Freedom: Raising Issues 


Finally, design freedom can be used to identify situations where decisions are 
implied by other actions. The plot below shows a narrow band in which the overall 
stiffness of the force feedback mouse linkage is under the designer’s control. Below a 
certain part count, the designer has no freedom to choose stiffness; likewise above a 
certain level the freedom of stiffness is minimal. The loss of design freedom can be 
indicated to the designer without having any previous notion about the value associated 
with system stiffness - the loss of freedom along this dimension is enough to warn the 
designer about and implicit decision being made. 


Design Freedom: Raising Issues 



As we minimize part count, we make an implicit 
decision about the stiffness of the mechanism 
joints -> Does the designer care? 
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Summary 


In summary, DBCD has been presents as a fundamental shift in the way design is 
viewed: away from a segregated view of requirement and attributes, toward a unified 
view in which any descriptive attribute for a design can become a requirement. This 
viewpoint is implemented in a probabilistic, mathematical framework that then yields to 
normative methodologies from decision theory: EVDM for selecting among the 
discrete, classification variables, and EVPI for refining uncertainty in the continuous, 
performance variables. Extending this process beyond typical decision-based design, 
measures of design freedom can help the designer to control the design process. Of 
particular note here is that the process of design does not preclude the generation of 
design space - measuring where and when it is lost can help to seed the creative process 
at the heart of design. 


DBCD provides a normative methodology for conceptual design whose fundamental 
purpose is to support the design process. It provides aids for navigating among design 
abstractions, setting design targets, and preserving design space. 


Summary 

• A fundamental shift in the view of design space: 
commingled requirements and attributes 

• A mathematical framework for modeling design space 

• Normative Set-Based Design 

- EVDM: Selecting Designs 

- EVPI: Refining Requirements 

- Design Freedom: Managing Design Space 

• Beyond Decision-Based Design 

- Designers can create design space 

• Where 

• When 

• How 
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Future Challenges in Innovation Practices: 
A View From Engineering Design Research 


Ade Mabogunje 
Center for Design Research 
Stanford University 
Stanford, CA 
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FUTURE CHALLENGES IN INNOVATION PRACTICES: 
A VIEW FROM ENGINEERING DESIGN RESEARCH 


In a time when access to information is unrivaled compared to earlier times, have we 
become more creative as engineers or more conservative? Are today's engineers in the 
aerospace industries making more discoveries and innovations today when compared to 
engineers in the 1960s? What are the changes in work spaces, work practices, and 
management practices between then and now? It has been suggested that the 
information revolution is in reality more of a control revolution, and as managers we 
have unconsciously become more controlling as our knowledge of the domain has 
shrank relatively to the relevant knowledge. Bringing in a project on time and under 
budget has made the calendar and the spreadsheet about the only tools we understand. 
But management is not the only one to blame. As engineers we have become simply 
overwhelmed by the amount of important and relevant information that is out there and 
all too conscious of the limits of what we can do within the constraints of any given 
project. Is there a way out? 
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A CASE FOR ACCELERATING INNOVATION 


Can we accelerate the rate at which we introduce new product and services? Is there 
a need for such acceleration? Do we have the know-how to do such a thing? 


A Case for Accelerating Innovation 

Is it possible to improve the rate at which we introduce new 
products and services? 
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THE NATURE OF DESIGN ENGINEERING 


For the purposes of this presentation: 

Forget for a while the numerous presentations and discussions about software and 
algorithms for design optimization. Let go of your association with the aerospace 
industry. Imagine you are the sole survivor of a shipwreck, and you have managed to 
swim to a small deserted island. What would you do? 


Contextual Questions 

What does design mean to us? What is design research? What is our approach to design 
research? 


Our response to these questions will reveal several important features of design. It is 
a natural human instinct, some authors have argued that it is at the core of engineering, 
it is best to think of design as encompassing R&D and manufacturing. Design embraces 
thinking from art to science, and it is best “optimized” through competition and 
selection. 


Design 

• A natural human instinct (...homo innovaticus). 

• The core of engineering thinking. 

• Encompasses thinking from R&D and Manufacturing. 

• Embraces thinking from art to science. 

• Best “optimized” through competition and selection. 

Science of artificial = Science of engineering 
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DESIGN RESEARCH 


Design Research as a field of research in science and engineering is relatively young. 
The focus of our work is on ways to improve the creativity of engineering design and 
the effectiveness of product design and development. 


The primary goal of NSF's Design 
Theory and Methodology Program is: 

• ...to create a new generation of fundamental principles 
and generic methods which will enhance the design 
process and provide the basis for educating a new breed 
of design engineer. 

• The results of fundamental research in this field can 
enhance design practice, improve design support 
systems, and strengthen the education of both practicing 
and future designers. 

• ... the results of research in this field can improve the 
creativity of engineering design and the effectiveness of 
product design and development, leading to improved 
national productivity and international competitiveness. 
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STANFORD DESIGN RESEARCH APPROACH 


The Stanford Center for Design Research was founded in 1983, in response to a 
growing realization that Design was poorly understood in industry and academia. In 
industry, for example the Automobile industry, there was a great loss in market share to 
Japanese car manufacturers. In the university, very few courses existed that taught 
engineers how to do design. 


At the Stanford Center for Design Research, our response was to pose two 
fundamental research questions: 

What do engineers do when they do design? 

How can we improve their process? 
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ENGINEERS OBSERVING ENGINEERS: REFLECTIVE 
AND REFLEXIVE RESEARCH 


Thus in response to earlier scenario I described where you were the sole survivor of 
a shipwreck, we would be interested in: 

what you would do (incl. why and how) 
what you could do 

what someone else in your situation would do 

We use various observational techniques including video to observe Engineers and 
Designers at Work. Our primary concern is with the nature of their thinking, the 
computational tools they use, and the social protocols they develop to better meet the 
challenges they face. 


video interaction analysis 

the power of scientific observation 
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MANAGING COMPLEXITY THROUGH SCALING 
AND DIMENTIONAL SIMILARITY 


Engineering Design is a complex process. The technical artifact in most aerospace 
companies is complex and the organizations are complex. One approach to managing 
this complex whole is to think about them as separate entities. This approach works 
when the product and processes are well known, and the desire of the organization is 
that of reproducing previous work. 


For innovative organizations, thinking of the artifact and the organization separately 
can be done artificially but is inefficient in practice. As in most engineering systems, 
the scaling approach will be more appropriate here. Thus our research practice is to 
observe the artifact and the organization at different scale levels, where the dimensions 
are level of complexity of the artifact and level of control in the organization. Each of 
these levels is called a test-bed. 


Studying Design Teams at Different Scale 
Levels - Test bed Approach 
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THE KEY TO ACCELERATING INNOVATION 


Our research suggests that the key to accelerating innovation is to focus on 
communication patterns in engineering design teams (See bibliography). In the rest of 
the presentation I will share with you sample material from research in our lab where 
we have systematically explored the fundamental questions I raised earlier and the 
results that have led us to this conclusion. 

I will structure the slides in the following manner: 

What do engineers do when they do design? Here I will present sample 
observations and early results 

How can we improve their process? Here I will present some of the tools, 
methods, and approaches. 

Towards the Innovative Organization? Here I will draw on extant literature to 
contrast Innovative organizations with Managerial ones. I will then indicate what our 
results have to say about ways to develop innovative organizations. 


Approaches to Improving Performance 

• Expert Systems 

• Augmentation Systems 

• Best Practices 

• Design Optimization 

• Decision support tools 

• Simulation-based Design 

• Industry Studies 

• Laboratory Studies 
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DESIGN PROCESS INSTRUMENTATION 


The focus on communication patterns of engineer has been embodied in an 
infrastructure for instrumenting the design process. This infrastructure , similar to a 
flight simulator allows us to observe the design process unfold in real-time and make 
suggestions for alternative courses of action through the use of expert design coaches. 

Among the parameters that can be monitored are those related to the product, such as 
the function, the features and the performance; and those related to the human system 
such as development time, product novelty, learning, experiences of pleasure, fun and 
satisfaction, and experiences of anxiety and frustration. 

By maintaining a dual focus on the human and technical side of product development 
we have been able to demonstrate a consistent pattern of exceptional design in one of 
the design test-beds we work with, the ME3 10 design course. 


Improving Process Performance through 
Design Instrumentation 


Design 

Product Activity 
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REFERENCES 


This list consists of the Stanford doctoral dissertations that in sum leads us to 
understand the critical role communication patterns of Engineers will play in the 
process of accelerating innovation. 
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Modeling Engineering Design Thinking and Performance 
As a Question Driven Process 


Ozgur Eris 

Center for Design Research 
Stanford University 
Stanford , CA 
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Design teams transform product requirements into product specifications. 
Traditionally, as a performance dimension, engineering design teams are trained 
to focus on the product. They identify and monitor key performance variables 
associated with the product. This is the basis for iteration in the design process. 


Traditional Design Performance Model 
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This video clip is an excerpt from a ABC network television program on the 
leading international design consultancy firm IDEO. The founder, Professor 
David Kelly, strongly emphasizes their expertise in the “design process” as a 
performance dimension over their expertise in a specific technical domain. 


IDEO Video 


D> 
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If David Kelly’s point is valid, then how do we account for the performance 
dimension that is associated with the design process? 


Traditional Design Performance Model 

Process Measurement? 
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One way of accounting for the performance dimension associated with the 
design process is to consider the human variables in design activity. By 
“human,” I am referring to the people who are directly involved in design 
activity and make up the design team, and not to the users. 


Traditional Design Performance Model 


Human 
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Identifying and monitoring human variables would result in “process 
measurement,” which can then be treated similarly to product measurement and 
constitute the basis for a second feedback/iteration loop. 


Human-Centered Design Performance Model 
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Identifying human performance variables is the main motivation behind this 
research. I conducted two sets of preliminary observations to accomplish this. I 
carried out the first set of observations at Ford’s Dearborn vehicle development 
center, and the second set of observations at Stanford during a graduate level 
mechanical engineering design course. 


During the observations, I initially paid attention to the decisions design 
teams were making. However, focusing on decisions resulted in an increased 
awareness of the influence of the questions that were being asked. Some 
questions seemed to influence pivotal decisions whereas other seemed to have 
no discemable impact and faded away. I attempted to create question-decision 
trees of design meetings in order to tease out such relationships. However, I 
quickly realized that our understanding of question asking processes of 
designers was limited, and decided to study the questions. 


Motivation/Initial Observations 



• Interplay between “Questions” and “Decisions” 

- Some questions have strong influence on pivotal decisions. 

- Initial Vision: Constructing question-decision trees in order to 
identify such influences. 
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On a conceptual level, the relationship between questions and decisions can 
be explained in terms of a duality: it is not possible to ask a question without 
making decisions, and to make a decision without asking questions. 


Motivation/Initial Observations 


A Duality between questions and decisions: 

• Every question operates on decisions as premises 
since questions are formulated (goal, content, 
structure, timing, etc.) Questions are intentional and 
not arbitrary. 

• Conversely, every decision operates on questions 

as premises since decision making entails dealing 
with choices that need to be generated, analyzed, 
and compared. 
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I made three key observations in the field, and built on them in constructing 
hypotheses related to the question asking behavior of design teams (will be 
discussed later in the presentation). 


Key Field Observations 

01 : The design teams spent a significant amount of time 
asking and discussing questions in order to: 

- reason about and explain phenomena, 

- generate and negotiate design concepts, 

- seek new information, 

- verify and clarify facts and each others views, 

- mediate social interaction. 

02: Meetings during which the teams asked more “good” 
questions yielded more progress (insights and 
discoveries). 

03: Working with existing artifacts and prototyping 
hardware had an effect on the nature of questions 
asked. 
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In order to test the hypotheses, I designed a quasi-controlled laboratory 
experiment, which required the operationalization (formalization) of three 
fundamental aspects of question asking. 


Characterization of Questions in the Lab 



Operationalizing questions in discourse involves formalizing: 

1. Definition 

2. Timing 

3. Nature 
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I identified questions in the verbal discourse of the design teams participating 
in the experiment according to the definition and criteria listed above. 


Definition of a Question in a Design Context 

Verbal utterance related to the design tasks 
at hand which demand explicit 
verbal and/or nonverbal responses. 

• A response constitutes an answer if it has been 
solicited by the questioner — a response which was 
not explicitly solicited does not constitute an answer. 

• What is directly observable: Communication of the 
question as opposed to its “occurrence.” 
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Formalizing the timing of a question entailed identifying the start and end of 
its communication. 


Timing of Questions 


• What can be observed directly 

- The start and the end of the communication of a 
question. 

• What can be studied 

- Temporal relationships to other questions, i.e. 
Frequency, Progression and Interplay. 
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In order to formalize the nature of a question, I relied on published 
taxonomies of questions, which date back to Aristotle’s Posterior Analytics . If a 
taxonomy abides by a unique differentiating principle, and is comprehensive, it 
can serve as a meaningful analysis scheme for coding the questions identified in 
discourse. 


In this slide, four such taxonomies are inserted into the columns of a table. 
Categories which are semantically similar are placed in the same row. 
Aristotle’s four fundamental classes are taken as the basis for the comparison. I 
would like to convey the following two points with this table: 

1) The four taxonomies map onto each other. Therefore, any one of them can be 
taken as an initial coding scheme. 


2) Graesser, after extending Lehnert’s framework, used the resulting taxonomy 
to code questions he identified during tutoring sessions. He discovered a 
correlation between a class of questions, which he called “Deep Reasoning 
Questions” and learning performance (as measured by a test score after the 
tutoring session). Therefore, it made the most sense to use his framework as an 
initial coding scheme. 


Review of Taxonomies of Questions 


ARISTOTLE 

DILLON [84] 

LEHNERT [78] 

GRAESSER [94] 

Existence 

(Affirmation) 

Existence/affirmation 

Verification 

Verification 

Instance/identification 


Nature 

(Essence/Def.) 

Substance/definition 


Definition 



Example 

Fact 

(Attribute/ 

Description) 

Character/description 

Feature Specification 

Feature Specification 

Concept Completion 

Concept Completion 

Quantification 

Quantification 

Function/application 

Goal Orientation 

Goal Orientation ■ 

Rationale/explication 

Concomitance 

Disjunctive 

Disjunctive 

Equivalence 


Comparison 

Difference 

Reason 

(Cause/ 

Explanation) 

Relation 


Interpretation 

Correlation 



Conditionality 
& Causality 

Causal Antecedent 

Causal Antecedent ■ 

Causal Consequent 

Causal Consequent* 

Expectational 

Expectational ■ 


Procedural 

Procedural ■ 

Enablement 

Enablement ■ 


■ Deep 
Reasoning 
Question 
(DRQ) 
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When I used Graesser’s taxonomy to categorize the questions asked by the 
design teams, I was not able to categorize 15-20 percent of the questions, which 
were rather influential. After considering these questions in depth, I realized that 
this could be because the driving premise of a design situation is rather unique 
and might not have been a factor in the formulation of the reviewed taxonomies 
(the four taxonomies reviewed earlier are epistemological approaches and are 
concerned with “what we know.”) 


Extending the Taxonomies of Questions 

• Underlying assumptions of the Taxonomies: 

- A specific answer, or a specific set of answers, exist 
for a given question. 

- Lehnert and Greaser also seem to assume that the 
answer is known. 

• Driving premise of a Design situation: 

- Multiple alternative known answers as well as 
multiple unknown possible answers exist. 

- The questioner’s intention is to disclose the alternative 
known answers, and to generate the unknown 
possible ones. 
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Further analysis of the questions I could not categorize led me to construct 
five new questions categories. I call these types of questions “Generative Design 
Questions.” (Each question category will be discussed in detail.) 


Extending the Taxonomies of Questions 


ARISTOTLE 

DILLON [841 

LEHNERT [781 

GRAESSER [941 

ERIS 
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Feature Specification 

Concept Completion 

Concept Completion 

Concept Completion 

Quantification 

Quantification 

Quantification 
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Goal Orientation ■ 

Rationale/Function ■ 

Rationale/explication 

Concomitance 
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Disjunctive 

Disjunctive 
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Difference 

Reason 

(Cause/ 
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Relation 


Interpretation 

Interpretation ■ 

Correlation 


1 

Conditionality 
& Causality 

Causal Antecedent 

Causal Antecedent ■ 

Causal Antecedent ■ 

Causal Consequent 

Causal Consequents 

Causal Consequent ■ 

Expectational 

Expectational ■ 

Expectational ■ 


Procedural 

Procedural ■ 

Procedural ■ 

Enablement 

Enablement ■ 

Enablement ■ 


Proposal/Negotiations 
Enablement • 


Method Generation • 
Scenario Creation • 
Ideation • 


■ Deep 
Reasoning 
Question 
(DRQ) 

1 Generative 
Design 
Question 
(GDQ) 
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What differentiates Generative Design Questions from the rest of the 
question categories is that they reflect divergent thinking. The reviewed 
question categories reflect convergent thinking. 


Implications of the Extension 


Convergent vs. Divergent Thinking Modes: 

• When asking GDQs, the questioner can be seen to 
be diverging from the facts to the possibilities that 
can be generated from them. The answers are not 
expected to hold a truth-value. 

• When asking DRQs, the questioner can be seen to 

be converging from possibilities to the facts. The 

answers are expected to hold truth-value. 


17 


209 



I designed the experiment to test these four hypotheses. The most important 
one is the postulated relationship between the incidence of deep reasoning and 
generative design questions, and design team performance. 


Hypotheses Derived from Field Observations 


HI : Question tinning, type and content are descriptors of 

design process. They are informative enough to serve as 
a roadmap to the design thinking and processes of teams. 

H2: DRQ+GDQ asking rates of design teams can be taken as 
a design performance metric. 

H3: Working with hardware influences the question asking 
behavior of design teams. 

H4: There is significant correlation between the frequency of 
discoveries made by design teams and their performance. 
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Short description of the design scenario used in the experiment. It is relevant 
to note that almost all of the graduate students who were subjects had more than 
two years of industry experience. 


Experiment Description 

The Bodiometer Exercise: 

• Design and prototype a measurement device to 
measure the length of various body contours. 

• 36 mechanical engineering graduate students 
designing in teams of 3 for 90 minutes. 

• Half of the teams were provided with the prototyping 
hardware at the beginning, the other half, 
approximately 30 minutes into the exercise. 
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Description and illustration of the Proposal/Negotiation Generative Design 
Question category. 


Generative Design Questions 

GDQ Category 

Proposal/Negotiation 

Definition 

Questioner wants to suggest or negotiate 
a concept. 

Example 

How about attaching a wheel to the body? 

Significance 

Proposing an idea in the form of a question 
promotes consideration and feedback. 
Negotiation promotes synthesis. 
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Description and 
Question category. 


illustration of the Scenario Creation Generative Design 


Generative Design Questions 

GDQ Category 

Scenario Creation 

Definition 

Questioner wants to construct a scenario 
and consider possible outcomes. 

Example 

What if the device was used on a child? 

Significance 

Accounting for possible outcomes generates 
and refines design requirements. 


► 
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Description and 
Question category. 


illustration of the Method Generation Generative Design 


Generative Design Questions 

GDQ Category 

Method Generation 

Definition 

Questioner wants to generate procedures of 
achieving a specific goal. 

Example 

How can we keep the wheel from spinning? 

Significance 

Operating with a specific goal generates a 
set of methods for implementing concepts. 
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Description and illustration of the Ideation Generative Design Question 
category. 


Generative Design Questions 

GDQ Category 

Ideation 

Definition 

Questioner wants to generate as many 
concepts as possible without a specific goal. 

Example 

What can we do with magnets? 

Significance 

Operating without a specific goal frees 
associations and drives concept generation. 
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Description and illustration of the Enablement Generative Design Question 
category. 


Generative Design Questions 

GDQ Category 

Enablement 

Definition 

Questioner wants to identify as many 
resources as possible that enable a concept. 

Example 

What allows you to measure distance? 

Significance 

Identification of multiple resources promotes 
surveying and learning from existing design 
features. 
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In order to test the hypothesis outlined earlier which postulates a relationship 
between the incidence of deep reasoning and generative design questions, and 
design team performance: 


1) All deep reasoning and generative design questions that were asked by the 
twelve design teams were identified. 


2) Performance of each team was measured by using two independent methods: 

i) A score indicating the degree to which each team met a set of design 
requirements related to accuracy, manufacturability, usability and price that 
were provided at the beginning of the experiment. 

ii) A subjective rank ordering of the prototypes by three Stanford mechanical 
engineering faculty. 


These two methods yielded results that correlated strongly. Therefore, only 
the first metric was used for evaluation. The combined deep reasoning and 
generative design question asking rate of each design team was plotted against 
its score. A linear correlation is visible. 


Note: There was a control and a test group in experiment. In the interest of 
time, I will not discuss the difference between them. For a detailed discussion, 
please see the following reference: 

Eris, Ozgur. Effective Inquiry for Innovative Engineering 
Design, Kluwer Academic Publishers, Boston, 2004. 


Question Asking vs. Performance 



DRQ+GDQ asking rate (questions/hr.) 
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Statistical analysis yielded strong correlation between the incidence of GDQs 
and DRQs and performance. There was no significant correlation when all of the 
questions or just the DRQs or the GDQs were considered. This suggests that 
deep reasoning and generative design questions are strongly related and need to 
be considered in conjunction. 


Question Asking and Performance 


Statistical Analysis: 



Control R 2 

Test R 2 

Control P 

TestP 

GDQ+DRQ vs. Score 

0.68 

0.70 

0.027 

0.023 

All Questions vs. Score 




0.110 

DRQ vs. Score 

0.45 


0.087 


GDQ vs. Score 


0.56 


0.054 


DRQs and GDQs need to be treated as 

Complementary Pairs 

when treating them as a measure of performance. 
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Asking deep reasoning and generative design questions can be treated as a 
mechanism for managing convergent and divergent thinking modes during 
design activity. 


Question Asking as a Mechanism for 
Managing Convergent-Divergent Thinking 

• During conceptualization, GDQs are instrumental in 
preserving or increasing conceptual ambiguity by: 

- reframing existing understandings that establish context, 

- generating alternatives, 

- creatively negotiating design concepts. 

• During implementation and assessment, DRQs are 
instrumental in reducing conceptual ambiguity by: 

- reiterating goals, 

- focusing on deliverables, 

- seeking and establishing causality, 

- reducing the number of alternatives. 

27 


219 



High performance design teams realize the importance of managing 
conceptual ambiguity, and use the GDQ and DRQ instruments in a balanced 
fashion to operate at the necessary level of abstraction throughout the design 
process. Therefore, the manifestation of convergent-divergent thinking in the 
question asking and decision making processes of design teams in the form of 
Deep Reasoning and Generative Design Questions constitutes a performance 
dimension in design activity. 


The resulting design thinking model illustrates the transformation of design 
requirements into design concepts through Generative Design Questions, and 
the transformation of those concepts into design decisions and specifications 
through Deep Reasoning Questions. 


Question-centric Design Thinking Model 



Design Concepts 
Cl, C2, C3, C4, C5... 
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Two potential applications are: constructing a method which promotes 
effective question asking and using it as a pedagogical tool, and treating the key 
findings of this research as requirements for the design of engineering design 
information systems. 


Applications 


• Pedagogical: a methodology to inform students of 
the importance of questioning in design thinking 

• Design Information Systems: 

- Interacting with the Knowledge System = 

Asking Questions of the Knowledge System 

- Capturing and retrieving Design Rationale 
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Design Education for the Aerospace Workforce 


George A. Hazelrigg 

Division of Design, Manufacture, and Industrial Innovation 
National Science Foundation 
Arlington, VA 
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Design is what defines engineering as an activity separate from the sciences. 
Yet design, while practiced for millennia, is perhaps the least well defined of the 
“engineering sciences,” and many people would insist that it is not a science at 
all. I want to take a few minutes to put design in perspective and discuss where 
we might go with design education. 
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To be sure, engineering educators and practicing engineers alike cannot agree 
on what is design. Some people feel that an important element of design is the 
generation of alternatives, some insist that design is creativity, others look upon 
design as a process of graphics and drawing (now really CAD), while others add 
to that dimensioning and tolerancing. Certainly, design would include some 
elements of product specification and planning. And I certainly would argue 
that design involves a good deal of selection or choice, that is, it involves a good 
deal of decision making. We could argue that design encompasses at least this 
circle of activities, and perhaps more. 


What is design? 

• The generation of alternatives 

• Creativity 

• Graphics/drawing 

• Dimensioning/tolerancing 

• Product specification 

• Planning 

• Selection/choice (decision making) 
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Now, what are the characteristics of design? As we have seen by the 
presentations so far, design presents an overwhelming set of possibilities, and 
dealing with this set is one of the things that makes design such a complex 
activity. But, in addition, the designer usually faces what appear to be 
conflicting goals, the presence of many actors — design is done by teams — and 
always a great deal of uncertainty. I’ll come back to the issue of uncertainty. 
The underlying question is then, how do you wade through this swamp to create 
a good product or system? 


What are the characteristics of design? 

• An overwhelming set of possibilities 

• Conflicting goals 

• Many actors 

• Uncertainty 

How do you wade through this swamp 
to create a good product or system? 
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Uncertainty is pervasive in all of design. You see, design is a matter of 
making choices in the present to achieve goals or desires set for the future. And 
you simply cannot predict the future with certainty. Think about what it would 
mean if you could. For an airplane, you would know when it would fly, and 
when it would be broken, how many passengers would be on each flight, when it 
would crash, how and why, and how much money the producing company 
would make as a function of its design. So the implications of all design 
decisions would be clear. Outcomes would be assured, and it’s hard to think of 
a reason for poor design decision making. But life just isn’t like this. All real 
decisions — design decisions included — are under uncertainty. 


What is the meaning of uncertainty? 

•You cannot know the future with certainty 

• If you could: 

-Success/failure would be obvious 
-Implications of choices would be clear 
-Outcomes would be assured 

-There would be no technical reason for poor 
design decision making 

• But all real decisions are under uncertainty 
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Recognizing that design is the defining element of an engineering education, 
let’s take a look at what we teach. Our engineering curricula include a good 
deal of mathematics, mainly focused on calculus and differential equations — the 
mathematics for modeling physical systems. We provide some 20-25 credits of 
math. Then, of course, we have the engineering sciences, which begin with 
fundamental courses in chemistry, physics and biology, and proceed into the 
more specialized sciences such as statics, kinematics, thermodynamics, strength 
of materials, and so on. Students get some 70-80 credits of the sciences. We 
sprinkle in some electives — not much in the usual engineering curriculum as 
there isn’t much time, maybe 12 credits. And finally, we have a capstone design 
course — 6 credits. This is what interests me: design is what engineering is all 
about, design is what defines engineering, design is the main reason why 
students enter engineering, and we devote less attention in the engineering 
curriculum to design than we do to the electives. 


What is the focus of engineering 
education? 

• Mathematics (calculus, differential eqs.) 

• Engineering sciences (chemistry, physics, 
biology — strength of materials, 
thermodynamics, circuit theory,. . .) 

• Electives (history, music, art,...) 

• Design (typically a capstone design course) 

Design gets about as much attention as electives. 
But, isn’t design what engineering is all about? 
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We can classify what we teach into four broad categories. We can teach 
skills: playing piano, riding a bicycle, teamwork, communications skills and so 
on. We can teach knowledge: the laws of nature, facts, relationships and 
causality. We can teach judgment. And then there’s creativity. I have question 
marks on creativity, as I have my doubts about teaching it. I feel that we really 
can’t define creativity clearly, we don’t know it when we see it, we have no 
validated pedagogy for creativity education, and we don’t know how to test for 
it to see if we have done any good at teaching it. About the only thing I have 
heard agreement on is that we are pretty good at unteaching it. So let me leave 
creativity out of this discussion. 


Skills and judgment are gained through experience, often through one-on-one 
tutoring. Knowledge is gained by thinking and studying. Knowledge is what we 
teach in lecture classes, and it is taught mainly by reducing concepts to theories 
and then presenting these theories and their applications. 


What do we teach? 

• Skills (communications, teamwork, 
interpersonal,...) 

• Knowledge (fundamentals, basics, science) 

• Judgment (experience) 

• Creativity (???) 

Skills and judgment are gained by doing and experiencing. 
Knowledge is gained by thinking/studying. Creativity is 
elusive. We don’t know how to define it, teach it, test it. 

But there is general consensus that we can unteach it. 
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The bottom line here is that skills and judgment are hard to teach. Teaching 
them can take quite a bit of time — even decades. It can require special 
equipment and tutoring, and it can be very expensive. To train an airline pilot 
may cost $2-3 million, for example. But the reward is that people who have 
skills and judgment are valuable, and they are often quite well paid. The 
question is, could we afford to spend something in excess of a $ 1 million to train 
each engineer? Then, could we afford to pay them well enough to justify this 
educational expense? 


An observation 
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Now, despite all that I have heard, even from academic department chairs, I 
want to make one thing perfectly clear, design is not the mindless application of 
the engineering sciences. In my mind, design encompasses much more than the 
engineering sciences. It is more than a multi-disciplinary activity, it is an omni- 
disciplinary activity — it includes all conceivable disciplines. 


What design is not 


Design is not the mindless 
application of the engineering 
sciences. 
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One of the things that designers believe that they have to deal with is the 
issue of conflicting goals. This is a notion that has stymied progress in design 
for some time. First, let me say that there are really only two kinds of decision 
making: decision making that is essentially based on optimization — the choice is 
directed by the desire to maximize against some preference, such as, I want to 
make money and more is better; and a process of chaos — essentially decision 
making by flipping a coin. There really is nothing in between. To understand 
this, it is important to note that decisions are made only by individuals. We’ve 
all heard statements such as these that refer to the chaos that results from group 
interactions. Groups, in fact, don’t “make” decisions. Rather, they have an 
emergent behavior that is the result of the decisions of the individuals who 
comprise the group, and the rules by which the members of the group interact. 
The field of mathematics that deals with group processes is called game theory. 
It was introduced by the famous mathematician John von Neumann and the 
economist Oskar Morgenstern in their seminal book "The Theory of Games and 
Economic Behavior .” I’ll come back to this shortly. But suffice to say that all 
group interactions are, to some extent, chaotic. 


The problem of conflicting goals 

• Two decision making alternatives 

-Optimization — choice directed by preference 
-Chaos — largely random 

• Decisions are made only by individuals 

-Too many chefs spoil the broth 

-A camel is a horse designed by a committee 

• Group interactions or group processes are 
referred to in mathematics as “games” 
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So, if groups always behave somewhat chaotically, why do we use groups in 
engineering design? Well, first of all, the job is just too big for one person. One 
person won’t live long enough to design every piece of a jumbo airliner. And 
one person doesn’t have the knowledge to do it successfully anyway. But more 
than this, designs often have to satisfy a group — consumers, for example — and it 
seems logical that the design process could benefit from the insights of a group 
of designers. 


The need for group processes in 
engineering design 

• One person can’t do it alone 
•We have to satisfy a group 

-Insights gained from members of the group 
should help, right? 
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But there are severe pitfalls of group processes that have important, and often 
negative, impacts on engineering design. First, although it might not seem so, 
unless a group is led by a dictator, the results of the group process are essentially 
random, that is, they are chaotic. This is hard to comprehend, especially for 
people who believe that they have recently had a “good” group experience. 
After all, many group processes seem so well organized and directed. But, 
clearly, the results of a group interaction depend critically on the rules of the 
interaction. Just think of a sports event. Team A wins, team B loses. But the 
result could have been quite the opposite if some little rule — a penalty rule, for 
example — had been different. Of course, you might think, what we need is to 
assure that group interactions simply proceed under a good set, the “right” set, of 
rules. This is the key issue — the rules — and it is addressed by Arrow’s 
Impossibility Theorem. Kenneth Arrow is a Stanford economist who won the 
1972 Nobel Prize in economics for the work he did in his PhD thesis in 1951. 
His work has been well vetted by a number of highly qualified experts, and it 
really bears paying attention to. 


The pitfalls of group processes in 
engineering design 

• Unless the group is led by a dictator, the 
results are essentially random — chaotic 

-This is hard to comprehend — group processes 
often appear well organized and directed 

• Clearly, the results of a group interaction 
depend upon the rules of the interaction 

• The key issue is addressed by Arrow’s 
Impossibility Theorem 
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Arrow focused on voting as the key group process. But voting is the essence 
of choice in a group process. It is how groups develop their emergent behavior. 
So Arrow postulated that the rules by which the group evolves its behavior 
should have certain rather obvious characteristics: (1) If everyone in the group 
prefers alternative A, then the group should choose A. (2) If the group agrees 
that A is preferred to B, and the group agrees that B is preferred to C, then the 
group should agree that A is preferred to C. (3) If the group is asked to express a 
preference between A and B, the result should not depend on the presence or 
absence of alternatives C or D or E. And, finally, (4) that more than one person 
in the group should have a say — that there is no dictator in the group who gets to 
make all the decisions irrespective of the preferences of the others in the group. 
These characteristics are pretty straight forward. The first seems obvious by 
examination. The second is needed for the group to be able to agree on a choice. 
If A is preferred to B and B to C, but C is preferred to A, there is no best choice 
for the group. Every alternative possesses a better alternative. And the third 
condition is needed to affect any choice, because additional alternatives, 
however irrelevant, can always be added. So these characteristics are really 
basic to rational group behavior. Without them, a group cannot behave in a 
rational way. 


What are the right rules for a group 
interaction? 

• Principles: 

-If everyone in the group wants A, the group 
chooses A 

-If the group would choose A over B, and B over 
C, the group would also choose A over C 

-If the group is asked to choose between A and B, 
the choice does not depend on whether C (or D 
or E) is also on the table 

-Everyone in the group gets a say (no dictator) 
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But Arrow’s Theorem provides us with the alarming truth: any set of 
rules — ANY SET — that satisfies the first three characteristics is of necessity a 
dictatorship. That is to say, no set of rules exist or can be found to exist that 
enable a group to have rational behavior. All group interactions are, in this 
sense, chaotic. What this means is that the behavior of a group depends as much 
on the rules of the interaction as it does on the individuals who make up the 
group, their preferences, and what they bring to the group. Optimality for the 
group is frequently not definable and, even when it is, no set of rules exist to 
assure that the group’s behavior is, in any sense, optimal. 


This has profound implications for engineering design, because what it says 
is that one can never be sure that a design that is the result of “teamwork” is the 
best possible design, the worst possible design, or anything in between. Indeed, 
one can be pretty certain, especially given the myriad of design possibilities, that 
any group design will be well off optimal. This is a strong case for dictatorial 
design, or at least for making a strong attempt to minimize the group-choice 
aspect of design. 


Arrow’s Impossibility Theorem 

• Any set of rules that satisfy the first three 
principles is of necessity a dictatorship 

• There is no set of rules for a group 
interaction that guarantees a desired outcome 

• All group interactions are, in some sense, 
chaotic 
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I know that, for all of you who have had great teamwork experiences, this 
stuff is really hard to believe. So I need to leave you with an example. Suppose 
a team of 100 people are trying to decide among five alternatives: what color 
should we design this thing — red, green, blue, orange, yellow (A, B, C, D, or E)? 
Let’s say the preferences of the 100 individuals are as given: for example, 45 
people like these in the order A, E, D, C, B (A is best, B is worst). 25 prefer the 
order BEDCA, 17 prefer CEDBA, and 13 prefer DECBA. Now, if the group 
decides to pick their preferred color by a one -person-one-vote procedure, it’s 
easy to see that A is the winner by a wide margin. Of course, everyone would 
walk away happy with the process, and confident that it was a good process, 
although many people, as individuals, would be dissatisfied with the result. On 
the other hand, if the group decides to throw out the worst color before moving 
on, then clearly A would be thrown out. Or, if the group decided to allow each 
person two votes (vote for the two colors you like best), then E would be the 
winner. You see, the winner depends on the voting rule used, not upon the 
preferences of the people voting. And you can verify for yourself that it is 
actually easy to find a voting rule under which any of the five alternatives is the 
winner, all without changing the preferences of the voters. The choice of voting 
rule introduces chaos in the result. But — Arrow’s Theorem — there is no correct 
voting rule. So you are doomed to endure chaos if you want a group process. 


Example of chaos in a group process 

• Choose among A, B, C, D and E 

- Individual preferences: 

• 45 people: A E D C B 

• 25 people: BEDCA 

• 17 people: CEDBA 

• 13 people: DECBA 


It is easy to find a rule 
under which any of these 
alternatives is chosen 


Rule 3: One person, two 
votes 

Result: A 45, B 25, C 17, 
D 13, E 100 


Rule 1 : One person, one vote 
Result: A 45, B 25, C 17, D 13, E 0 


Rule 2: One person, one vote for the 
worst (throw out the loser) 

Result: A 55, B 45, C 0, D 0, E 0 
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Now let me come back to uncertainty. Uncertainty is a term that we use to 
refer to events that are in the future, and only to events that are in the future. 
Probability theory is the mathematics of uncertainty. Because probabilities 
relate to events that are in the future, we have no data on them — EVER! Ergo, 
ALL PROBABILITIES ARE SUBJECTIVE. There just is no other kind of 
probability. Probability estimates are based on judgment, not on data. Data may 
be used to influence our judgment, but the data are never on the event in 
question. They may be on similar events, but never on the future event. What 
this means is that it is futile to say, “I don’t have the necessary data to estimate 
that probability.” You never do and never will. What you need is judgment, 
and judgment comes from experience. 


So, engineering design involves dealing with uncertainty. Understanding of 
uncertainty comes from judgment, and judgment comes from experience. But 
our engineering curricula don’t focus on design experience, so we largely fail to 
teach judgment. 


Back to uncertainty 

• Uncertainty relates to events in the future 

• Probability theory is the mathematics of 
uncertainty 

• Probabilities are all subjective (there is only 
one kind of probability) 

• Probability estimates are based on judgment 

• Judgment comes from experience 
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I need to add a few words about probability theory. Probabilities behave in 
ways that are highly counterintuitive. What probability theory does is to enable 
us to think consistently about probabilities. It does not guide us in any way 
toward “solutions” for specific probability numbers. Another thing about 
probability theory is that it is closely linked to decision making. After all, the 
main reason for thinking about probabilities is that we care to make decisions, 
and we want our decisions to be consistent with what we know and what we 
want. Putting probabilities into the context of decision making imposes severe 
restrictions on them. I urge you to look up the Dutch Book. The Dutch Book is 
a bookie who makes bets in such a way that he always wins. And the Dutch 
Book mathematics shows that ONLY Kolmogorov probability provides a self 
consistent framework for decision making. ALL other approaches, such as 
fuzzy logic, fail the test of self-consistency in the context of decision making. 


Probability theory 

• Probabilities behave in ways that are highly 
counterintuitive 

• Probability theory helps us to think 
consistently about probabilities 

• Probability theory does not guide us to 
“solutions” for probabilities 

• Probability theory is tightly linked to 
concepts of decision making and decision 
theory 
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So, let’s start to wrap up our thinking here. The key to good design is good 
decision making. But all real decisions are under uncertainty and risk. So good 
decision making demands a good understanding of uncertainty and risk. A good 
understanding of uncertainty and risk comes from good judgment. Good 
judgment comes from experience, and gaining experience takes time and it is 
expensive. So, how can we educate good design engineers without it taking 
decades and costing a fortune? 


Good design 

• The key to good design is good decision 
making 

• Good decision making demands a good 
understanding of uncertainty and risk 

• Good understanding of uncertainty and risk 
comes from good judgment 

• Good judgment comes from experience 

• Gaining experience takes time and it is 
expensive 
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We have observed that the teaching of skills and judgment is time consuming 
and expensive, while the teaching of knowledge is relatively quick and cheap. 
Now the trick is that we can convert skills to knowledge through the 
development and teaching of theory. For example, one way to teach addition 
would be to present the student with column after column of numbers and ask 
the student to memorize the column and the results. But there are infinite 
columns of numbers to memorize, and the process will take eons. On the other 
hand, addition can be reduced to theory — a set of rules — which, when learned 
allows the student to do addition. And these rules can be quite compact and 
simple to learn. The same is true with design. The development of a theory of 
design, that is, a theory of design decision making under uncertainty and risk, 
and the teaching of this theory will obviate the need for much of the skill and 
judgment demanded by our current approach. In short, a unifying theory enables 
us to use one or a few case studies to impart the necessary skills and judgment 
that, without the theory, would take many cases and years of experience. 


An alternative 

• The teaching of skills and judgment is time 
consuming and expensive 

• The teaching of knowledge is quick and 
cheap 

• Skill can be converted to knowledge through 
the development of theory — knowledge of 
the fundamentals 

• Development of a theory of design will 
greatly enhance design education 
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What about judgment? Can we effectively teach judgment in a compact 
experience? I believe that judgment can be aided by the judicious use of data, 
and it might be trained by simulation. I would call for development of the 
necessary pedagogy to improve the teaching of judgment in the engineering 
curricula. 


What about judgment? 

•To some extent, judgment can be aided by 
the judicious use of data 

•To some extent, judgment might be trained 
by simulation 

• We need to develop necessary pedagogy 
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So, where do we start? Fortunately, we don’t have to develop any new 
theory. It already exists, and has for quite some time. The necessary 
mathematics include probability theory, decision theory, game theory, 
optimization theory, and microeconomics, to mention several. The problem is 
that engineers are generally not familiar with these mathematics, and they don’t 
see their applicability. So, engineers are not the people to turn to for the 
development of a theory of design. Let’s face it, engineers have had over 50 
years to develop a theory of design, and they haven’t done it yet. So, why would 
we choose to believe that they will in another 50 years. 


Where do we start? 

• The necessary mathematics for a theory of 
design exists: 

- Probability theory 

- Decision theory 

- Game theory 

- Optimization theory 

- Microeconomics 

• But engineers generally are not familiar with 
these mathematics 

• So, engineers are not the people to turn to for 
a theory of design 
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What we need to do is to embrace experts from other fields, such as 
mathematics and economics, and ask their help in developing a theory of design 
and supporting pedagogy. The challenge is to integrate extant theories from 
related fields into engineering design. Although this is straight forward, it is not 
trivial. It will take drastic revision in our thinking and wholly new ways of 
approaching design. For example, I contend that we will throw out the concept 
of requirements altogether, and replace them with preferences. Requirements 
offer no decision guidance whatever among alternative designs that meet the 
requirements. Preferences provide guidance across the full range of alternatives. 
Nor will it be trivial to implement a theory of design as it can be 
computationally intensive. Thus, support software will greatly aid in the 
necessary transition. 


How do we get there? 

•We need to embrace experts from 
mathematics and economics and other 
relevant fields 

• The challenge is to integrate extant theories 
from these fields into engineering design 

• Then, we need to develop the appropriate 
pedagogy 

• And we need software to support application 
of the theory 


245 



Nor is it the case that no progress has been made to date. An underlying 
theory of design has already been proposed. Called “decision-based design,” the 
theory is based on the mathematics of decision theory. This theory is slowly 
gaining acceptance, however, it leads to approaches that are far afield from the 
conventional, requirements-based approach, and it demands a full rethinking of 
our approach to design. The development of support software could well pace 
much of the transition to decision-based design. 


Progress to date 

• An underlying theory and framework has 
been proposed for “decision-based design” 

• It is slowly gaining acceptance 

• Its implementation will depend on the 
development of support software and the 
continued development of supporting 
theories (demand theory, simulation theory, 
optimization theory, etc.) 
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So, in my mind, the way to improve design education is to provide a theory 
of design that allows us to move design away from a skill and judgment-based 
activity and toward a knowledge-based activity. Not only will this improve our 
overall ability to realize good products and systems, but it will allow us to create 
an effective engineering education system that recognizes and promotes good 
design education. One of my personal goals has been to elevate the 
respectability of engineering design as a legitimate “engineering science,” and I 
look forward to the day when the emphasis on engineering design in the 
engineering curricula matches the emphasis we currently place on electives. 


The end goals 

•To improve our overall ability to realize 
good products and systems 

•To create an effective engineering education 
system that recognizes and promotes good 
design education 

•To enhance the respectability of engineering 
design as a legitimate “engineering science” 

•To elevate engineering design above 
“electives” in the engineering curriculum 
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Teaching Design Across Disciplines 


Blaine Lilly 
Ohio State University 
Columbus, OH 
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I mention my background here because I believe it’s relevant to how I 
approach teaching. My experience as an apprentice tool and die maker at 
General Motors 25 years ago has affected how I view teaching and learning. My 
goal as a teacher has been to try to bring part of the “one on one” apprenticeship 
experience into my classroom in one of the largest universities in the U.S. 


Background 

■ Tool and die maker, General Motors 

■ BS and MS in mechanical engineering 

■ PhD in industrial and systems engineering 

■ Biases: 

■ Undergraduate education is crucial 

■ Manufacturing and design are inseparable 

■ “Hands on” experience is invaluable 
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This slide is just intended to give some idea of the size of Ohio State, which 
is one of the largest campuses in the country. While we are not one of the elite 
schools, we nevertheless turn out thousands of engineers who go on to careers in 
industry, government, and academia. 


More than football: 

■ Ohio State is large: 

■ 50,000+ students on Columbus campus 

■ College of Engineering: 

■ 5500 undergraduates - 1 5% of OSU total 

■ 1500 graduate students - 80% international 

■ Sixteen degree-granting programs 
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Like most state-supported schools over the past decade, Ohio State is 
struggling to maintain our quality in the face of relentless annual budget cuts. 
Our situation is made worse by the fact that we must compete in the legislature 
for funding with nine other state universities, each of which has a local 
constituency. As a result, although we’re the “flagship” university for the state, 
we do not have the same clout that a Purdue or a Penn State has in their 
respective states. 


The bigger picture 

■ State support has declined steadily 

■ FY 2003: $2.7 billion total budget 

■ State of Ohio provides $471 million 

■ Ohio has 9 other state universities that 
compete for funding. 
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This slide and the next slide are merely intended to provide context for what 
is happening in our region. While the numbers are for Ohio, the picture is much 
the same in Michigan, Illinois, Pennsylvania, and West Virginia. The traditional 
industrial heartland of the U.S. is dying, and very little is being done to reverse 
these trends. 


■ Ohio is experiencing a brain drain - many 
of our students must leave to find jobs. 

■ Core industries are going under 

■ Steel, rubber and glass are long gone 

■ Tool and die, aerospace and automotive are 
disappearing 

■ Outsourcing, lean thinking, health care 


254 



Manufacturing in Ohio 



179,000 jobs lost since July 2000 
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Here I attempt to make a connection between the environment I teach in and 
the role played by product design. I firmly believe that good product design is 
intimately related to a strong manufacturing economy. It’s impossible to do 
product design effectively without a good understanding of manufacturing 
processes. When a nation loses its manufacturing prowess, as England did in the 
1950’s, then technological prowess will also follow. The question for us as 
educators is, what can we do? 


Why does this matter? 

■ Product design and manufacturing are 
inseparable. 

■ If manufacturing leaves, design will follow 
- consider England vs. Germany. 

■ As educators, what should our response 
be? 
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The solution I’ve come up with is this: the U.S. clearly cannot compete in the 
“widget world”, where we try to manufacture low-cost goods competitively. 
China will own those markets for years to come. Our only hope is to concentrate 
on high-end products. Along with that, we need to train our young engineers to 
be open-minded, innovative, and aware that they can never stand still. We must 
make our students understand the world they’re entering, and the need to keep 
abreast of it. 


What’s the connection? 

■ The US engineering community is: 

■ Open 

■ Innovative 

■ Constantly changing 

■ Our students must be educated to thrive 
in this challenging environment. 
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I think our narrow, focused view of engineering must change. This is a tough 
sell to professors, who are rewarded for being specialists. Specialization is of 
course necessary for Ph.D’s, but we need to understand that at least 90% of our 
students will never go further than a Masters degree, at most. We need to expose 
our undergraduates to the tools they’ll need to survive in a world where breadth 
of knowledge is as important as depth. 


Challenges 

■ We must find ways to: 

■ Demolish the silos 

■ Expose students to the big picture 

■ Provide exposure to 

• Business 

• Economics 

• Language 

• Psychology 

• Industrial Design 
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This course has been developed slowly over the past ten years, and is the 
only course in the College of Engineering at Ohio State that deals with the 
issues I’ve mentioned. The course has been very heavily subscribed for several 
years, and has had consistently high evaluations from the students. 


ME/ISE 682: 

Fundamentals of product design 

■ Students from ME, ISE, EE, Aero, WE, Material 
Science, Architecture, Ind Design 

■ Seniors, first-year grad students 

■ Four credit hour technical elective 

■ 40 students per quarter, three per year 

■ 3 hours lecture, 2 hour lab per week 
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This slide is self explanatory. The overriding goal is to put what they’ve 
learned for four or five years into some logical context, and get the students to 
understand something of the history of engineering, and the importance of 
design. 


My approach 

■ Teach with artifacts 

■ Maximize hands-on experience 

■ Multi-disciplinary focus 

■ Relate to engineering history 

■ Fill the gaps 

■ Embed design in context 
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I use this example in many classes to point out the importance of constraints 
on design. In this case, the fact that 300 million cans are produced in the US 
every day leads them to understand how material costs can drive a design. A 
very good reference here is the article by Hosford in the September 1994 
Scientific American. 
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I have made extensive use of the Kodak single-use cameras in the classroom. 
The cameras are cheap, but are highly engineered objects. I’ve used these to 
teach freshmen, seniors, and NASA engineers, and every group can find 
something about the cameras to relate to. 


Uses for cameras: 

■ Product portfolios 

■ System architecture 

■ Design for manufacture 

■ Injection molding 

■ Intro to statistics - freshman engineering 

■ Lean assembly methods 
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Here we use hundreds of disassembled cameras in a lab to teach the students 
the principles of lean vs mass assembly. The students often have done similar 
exercises with paper airplanes, etc., but find that assembling real devices makes 
the exercise much more interesting. 


Lean assembly lab 
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I also believe that engineering students need to spend more time in the early 
stages of design, meaning conceptualization. I have worked with colleagues 
from Industrial Design to teach my engineering students how to sketch quickly 
in three dimensions for the purposes of communication and ideation. Even a few 
hours training can lead to quite a bit of improvement in these skills. 


Maximize “hands on” 

■ Students typically have little exposure to 
“hands on” design experiences. 

■ Curriculum is heavy on analysis, light on 
design 

■ I emphasize: 

■ Perspective sketching for ideation 

■ “Quick and dirty” prototype building 

■ Product disassembly and analysis 
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Example sketches from students after six hours of training. 
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Following the sketching exercise, the students build three dimensional 
prototypes from foam core and other cheap materials. The point here is to get 
them to see their ideas in three dimensions. I believe that this exercise is a 
necessary complement to learning CAD skills. 
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This slide is self explanatory, and these days, I’m preaching to the choir. I 
think it’s very useful to bring design and architecture students into engineering 
classes such as this one, because they show the engineering students that there 
are other valid ways to approach problems. The engineering students typically 
enjoy the interaction, as well. 


Multi-disciplinary focus 

■ Teaching to engineers, architects, and 
industrial designers can be constraining. 

■ Student teams are mixed by discipline. 

■ ID and architecture students add “yeast” 
to the student teams. 

■ Students typically enjoy the interaction. 
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A few brief thoughts on the discipline of industrial design. ID as a discipline 
has changed in the past decade or so, with much more emphasis placed on 
designing artifacts that work well and are environmentally sound, in addition to 
looking good. My interactions with industrial designers have been very positive. 


The ID perspective 

■ Emphasis on ideation and creativity 

■ Willingness to iterate early on 

■ “Get physical fast” 

■ Scenario modeling 

■ Cognitive engineering: human - machine 
interface 
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The original Mini, designed by Alex Issigonis. A fine example of designing 
within strict constraints. 


Engineering history 
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I see my product design course as an opportunity to plug some of the gaps in 
our program at OSU. Many tools and techniques current in industry are 
overlooked in the university, and here I list some of the tools that I expose the 
students to. 


Fill the gaps 

■ Concept generation and selection 
techniques 

■ QFD methods 

■ System architectures 

■ Functional decomposition 

■ Booth royd-Dewhurst DFMA 

■ Lean manufacturing 
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Where we intend to go from here. The next logical step is to develop a 
program in product design at the M.S. level, and institute a minor for 
undergraduates in design. 


What’s next? 

■ Create an interdisciplinary design 
program at the M.S. level 

■ Interdisciplinary minor in design 

■ Year-long senior design projects 
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